You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Yingchun Lai (Code Review)" <ge...@cloudera.org> on 2022/02/21 11:46:57 UTC

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Yingchun Lai has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18258


Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................

[metrics] Add table level metrics column_count and schema_version on master

This patch adds some table level metrics on master.
- column_count: The table's newest schema column count. Since too
  many column count will lead to bad performance and space
  amplification, we can add some monitor alerts based on this
  metric after this.
- schema_version: The table's schema version. Too many or too
  frequently schema changes is also a metric worth to be monitored.

Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
---
M src/kudu/master/catalog_manager.cc
M src/kudu/master/table_metrics.cc
M src/kudu/master/table_metrics.h
3 files changed, 19 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/58/18258/1
-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 1
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/18258 )

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................


Patch Set 9: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 9
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Fri, 18 Mar 2022 02:29:14 +0000
Gerrit-HasComments: No

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has removed a vote on this change.

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................


Removed Verified-1 by Kudu Jenkins (120)
-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 8
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18258 )

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................


Patch Set 9: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 9
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Fri, 18 Mar 2022 01:15:06 +0000
Gerrit-HasComments: No

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Abhishek Chennaka, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/18258

to look at the new patch set (#4).

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................

[metrics] Add table level metrics column_count and schema_version on master

This patch adds some table level metrics on master.
- column_count: The column count in the table's latest schema. Since too
  many column count will lead to bad performance and space
  amplification, we can add some monitor alerts based on this
  metric after this.
- schema_version: The table's schema version. Too many or too
  frequently schema changes is also a metric worth to be monitored.

Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
---
M src/kudu/master/catalog_manager.cc
M src/kudu/master/table_metrics.cc
M src/kudu/master/table_metrics.h
3 files changed, 19 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/58/18258/4
-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 4
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/18258 )

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/18258/4/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:

http://gerrit.cloudera.org:8080/#/c/18258/4/src/kudu/master/catalog_manager.cc@6669
PS4, Line 6669:       }
              :         metrics_->live_row_count->set_value(static_cast<int64_t>(live_row_count));
              :       }
              :     }
              :   }
              : }
> Seems we update these two metrics every time we process tablets' report fro
Good suggestion, done.



-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 5
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Fri, 11 Mar 2022 16:50:42 +0000
Gerrit-HasComments: Yes

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18258 )

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................


Patch Set 8: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 8
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Mon, 14 Mar 2022 14:21:45 +0000
Gerrit-HasComments: No

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/18258 )

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................


Patch Set 9: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 9
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Fri, 18 Mar 2022 02:29:37 +0000
Gerrit-HasComments: No

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18258 )

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/18258/4/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:

http://gerrit.cloudera.org:8080/#/c/18258/4/src/kudu/master/catalog_manager.cc@6669
PS4, Line 6669: {
              :     TableMetadataLock l(this, LockMode::READ);
              :     const SysTablesEntryPB& pb = metadata().state().pb;
              :     metrics_->column_count->set_value(pb.schema().columns().size());
              :     metrics_->schema_version->set_value(pb.version());
              :   }
Seems we update these two metrics every time we process tablets' report from tservers, which is a bit weird. What do you think of updating them as needed when processing DDL requests?



-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 4
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Fri, 11 Mar 2022 10:48:05 +0000
Gerrit-HasComments: Yes

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/18258

to look at the new patch set (#3).

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................

[metrics] Add table level metrics column_count and schema_version on master

This patch adds some table level metrics on master.
- column_count: The table's newest schema column count. Since too
  many column count will lead to bad performance and space
  amplification, we can add some monitor alerts based on this
  metric after this.
- schema_version: The table's schema version. Too many or too
  frequently schema changes is also a metric worth to be monitored.

Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
---
M src/kudu/master/catalog_manager.cc
M src/kudu/master/table_metrics.cc
M src/kudu/master/table_metrics.h
3 files changed, 19 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/58/18258/3
-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 3
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Yifan Zhang, Kudu Jenkins, Abhishek Chennaka, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/18258

to look at the new patch set (#5).

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................

[metrics] Add table level metrics column_count and schema_version on master

This patch adds some table level metrics on master.
- column_count: The column count in the table's latest schema. Since too
  many column count will lead to bad performance and space
  amplification, we can add some monitor alerts based on this
  metric after this.
- schema_version: The table's schema version. Too many or too
  frequently schema changes is also a metric worth to be monitored.

Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
---
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
M src/kudu/master/master-test.cc
M src/kudu/master/table_metrics.cc
M src/kudu/master/table_metrics.h
5 files changed, 86 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/58/18258/5
-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 5
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Abhishek Chennaka (Code Review)" <ge...@cloudera.org>.
Abhishek Chennaka has posted comments on this change. ( http://gerrit.cloudera.org:8080/18258 )

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................


Patch Set 9: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 9
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Wed, 16 Mar 2022 02:03:19 +0000
Gerrit-HasComments: No

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has removed a vote on this change.

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................


Removed Code-Review+2 by Yingchun Lai <ac...@gmail.com>
-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 9
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Yifan Zhang, Kudu Jenkins, Abhishek Chennaka, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/18258

to look at the new patch set (#6).

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................

[metrics] Add table level metrics column_count and schema_version on master

This patch adds some table level metrics on master.
- column_count: The column count in the table's latest schema. Since too
  many column count will lead to bad performance and space
  amplification, we can add some monitor alerts based on this
  metric after this.
- schema_version: The table's schema version. Too many or too
  frequently schema changes is also a metric worth to be monitored.

Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
---
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
M src/kudu/master/master-test.cc
M src/kudu/master/table_metrics.cc
M src/kudu/master/table_metrics.h
5 files changed, 90 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/58/18258/6
-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 6
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/18258 )

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................


Patch Set 8:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/18258/8/src/kudu/master/catalog_manager.h
File src/kudu/master/catalog_manager.h:

http://gerrit.cloudera.org:8080/#/c/18258/8/src/kudu/master/catalog_manager.h@369
PS8, Line 369: UpdateSchemaMetrics
> My another thought is these schema related metrics are really 'metrics' or 
These gauge type schema related metrics are a little similar to 'attributes' in some point-of-view, but I think the differences are, attributes are rarer to change than metrics, and not so convenient to display on monitoring systems if not do transformations.



-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 8
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Sat, 12 Mar 2022 08:10:02 +0000
Gerrit-HasComments: Yes

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18258 )

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................


Patch Set 8:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/18258/8/src/kudu/master/catalog_manager.h
File src/kudu/master/catalog_manager.h:

http://gerrit.cloudera.org:8080/#/c/18258/8/src/kudu/master/catalog_manager.h@369
PS8, Line 369: UpdateSchemaMetrics
My another thought is these schema related metrics are really 'metrics' or just 'attributes' of a MetricEntity? I think attributes can also be exposed to monitoring systems. Maybe num_schema_changes is more like a metric.



-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 8
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Sat, 12 Mar 2022 07:15:59 +0000
Gerrit-HasComments: Yes

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Abhishek Chennaka (Code Review)" <ge...@cloudera.org>.
Abhishek Chennaka has posted comments on this change. ( http://gerrit.cloudera.org:8080/18258 )

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................


Patch Set 8: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 8
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Mon, 14 Mar 2022 16:00:00 +0000
Gerrit-HasComments: No

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/18258

to look at the new patch set (#2).

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................

[metrics] Add table level metrics column_count and schema_version on master

This patch adds some table level metrics on master.
- column_count: The table's newest schema column count. Since too
  many column count will lead to bad performance and space
  amplification, we can add some monitor alerts based on this
  metric after this.
- schema_version: The table's schema version. Too many or too
  frequently schema changes is also a metric worth to be monitored.

Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
---
M src/kudu/master/catalog_manager.cc
M src/kudu/master/table_metrics.cc
M src/kudu/master/table_metrics.h
3 files changed, 19 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/58/18258/2
-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 2
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Yifan Zhang, Kudu Jenkins, Abhishek Chennaka, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/18258

to look at the new patch set (#7).

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................

[metrics] Add table level metrics column_count and schema_version on master

This patch adds some table level metrics on master.
- column_count: The column count in the table's latest schema. Since too
  many column count will lead to bad performance and space
  amplification, we can add some monitor alerts based on this
  metric after this.
- schema_version: The table's schema version. Too many or too
  frequently schema changes is also a metric worth to be monitored.

Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
---
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
M src/kudu/master/master-test.cc
M src/kudu/master/table_metrics.cc
M src/kudu/master/table_metrics.h
5 files changed, 90 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/58/18258/7
-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 7
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Yifan Zhang, Kudu Jenkins, Abhishek Chennaka, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/18258

to look at the new patch set (#8).

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................

[metrics] Add table level metrics column_count and schema_version on master

This patch adds some table level metrics on master.
- column_count: The column count in the table's latest schema. Since too
  many column count will lead to bad performance and space
  amplification, we can add some monitor alerts based on this
  metric after this.
- schema_version: The table's schema version. Too many or too
  frequently schema changes is also a metric worth to be monitored.

Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
---
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
M src/kudu/master/master-test.cc
M src/kudu/master/table_metrics.cc
M src/kudu/master/table_metrics.h
5 files changed, 90 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/58/18258/8
-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 8
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Abhishek Chennaka (Code Review)" <ge...@cloudera.org>.
Abhishek Chennaka has posted comments on this change. ( http://gerrit.cloudera.org:8080/18258 )

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................


Patch Set 3: Code-Review+1

(1 comment)

http://gerrit.cloudera.org:8080/#/c/18258/3/src/kudu/master/table_metrics.cc
File src/kudu/master/table_metrics.cc:

http://gerrit.cloudera.org:8080/#/c/18258/3/src/kudu/master/table_metrics.cc@41
PS3, Line 41: The table's newest schema column count
nit: Maybe rephrase it to:
The column count in the table's latest schema.



-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 3
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 01 Mar 2022 04:58:38 +0000
Gerrit-HasComments: Yes

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/18258 )

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/18258/3/src/kudu/master/table_metrics.cc
File src/kudu/master/table_metrics.cc:

http://gerrit.cloudera.org:8080/#/c/18258/3/src/kudu/master/table_metrics.cc@41
PS3, Line 41: The column count in the table's latest
> nit: Maybe rephrase it to:
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 4
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Tue, 01 Mar 2022 05:56:07 +0000
Gerrit-HasComments: Yes

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18258 )

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................


Patch Set 8: Verified+1

Unrelated test failure in ConcurrentRebalancersTest.TwoConcurrentRebalancers/1.


-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 8
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Mon, 14 Mar 2022 14:25:08 +0000
Gerrit-HasComments: No

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/18258 )

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................


Patch Set 9:

Rebased the master branch, help to review again, thanks!


-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 9
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>
Gerrit-Comment-Date: Tue, 15 Mar 2022 03:12:20 +0000
Gerrit-HasComments: No

[kudu-CR] [metrics] Add table level metrics column count and schema version on master

Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18258 )

Change subject: [metrics] Add table level metrics column_count and schema_version on master
......................................................................

[metrics] Add table level metrics column_count and schema_version on master

This patch adds some table level metrics on master.
- column_count: The column count in the table's latest schema. Since too
  many column count will lead to bad performance and space
  amplification, we can add some monitor alerts based on this
  metric after this.
- schema_version: The table's schema version. Too many or too
  frequently schema changes is also a metric worth to be monitored.

Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Reviewed-on: http://gerrit.cloudera.org:8080/18258
Tested-by: Kudu Jenkins
Reviewed-by: Abhishek Chennaka <ac...@cloudera.com>
Reviewed-by: Yifan Zhang <ch...@163.com>
Reviewed-by: Yingchun Lai <ac...@gmail.com>
---
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
M src/kudu/master/master-test.cc
M src/kudu/master/table_metrics.cc
M src/kudu/master/table_metrics.h
5 files changed, 90 insertions(+), 9 deletions(-)

Approvals:
  Kudu Jenkins: Verified
  Abhishek Chennaka: Looks good to me, but someone else must approve
  Yifan Zhang: Looks good to me, but someone else must approve
  Yingchun Lai: Looks good to me, approved

-- 
To view, visit http://gerrit.cloudera.org:8080/18258
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia0671cfb0758d53e950a4e35a968dae15f82d18e
Gerrit-Change-Number: 18258
Gerrit-PatchSet: 10
Gerrit-Owner: Yingchun Lai <ac...@gmail.com>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <ac...@gmail.com>