You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Attila Bukor (Code Review)" <ge...@cloudera.org> on 2020/04/07 14:33:40 UTC

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Hello Andrew Wong, Adar Dembo, Grant Henke, Hao Hao,

I'd like you to do a code review. Please visit

    http://gerrit.cloudera.org:8080/15671

to review the following change.


Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................

KUDU-3078 Refactor Sentry integration tests

This commit refactors the existing Sentry integration tests to prepare
for Ranger integration tests.

It changes HMS and Sentry integration test base classes to test
harnesses that don't inherit from Gtest to simplify inheritance when
typed tests.

Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
---
M src/kudu/integration-tests/external_mini_cluster-itest-base.h
M src/kudu/integration-tests/hms_itest-base.cc
M src/kudu/integration-tests/hms_itest-base.h
M src/kudu/integration-tests/master_hms-itest.cc
M src/kudu/integration-tests/master_sentry-itest.cc
M src/kudu/integration-tests/ts_sentry-itest.cc
6 files changed, 456 insertions(+), 190 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/71/15671/1
-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 1
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15671 )

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................


Patch Set 3:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_hms-itest.cc
File src/kudu/integration-tests/master_hms-itest.cc:

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_hms-itest.cc@80
PS2, Line 80:     hms_opts.enable_kerberos = EnableKerberos();
            :     hms_opts.service_principal = "hive";
            :     harness_.hms_client_.reset(new HmsC
> hm maybe I'm missing something, but how would I reset hms_client_ if I acce
It probably wouldn't hurt to move this into the harness itself with some HmsITestHarness::ResetHmsClient(address, opts) or somesuch.


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc
File src/kudu/integration-tests/master_sentry-itest.cc:

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc@414
PS2, Line 414:                             
> maybe I misunderstood what you're suggesting, but wouldn't that be duplicat
Would something like this work?

SentryITestHarness {
  /* looks like it does now for the most part */
  virtual ExternalMiniClusterOptions GetClusterOpts() {
    /* opts we have in SetUpCluster */
  }
  void SetUpCluster() {
    ...
    SetUpClusterWithOpts(GetClusterOpts());
    ...
  }
};

SentryITestHarnessWithCache : public SentryITestHarness {
  ExternalMiniClusterOptions GetClusterOpts() override {
    ExternalMiniClusterOptions opts = SentryITestHarness::GetClusterOpts();
    // add cache gflag
    return opts;
  }
};

It's a little more legwork, but it helps decouple the harness and test a bit, which I find appealing.



-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 3
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 07 Apr 2020 21:08:58 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15671 )

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................


Patch Set 8:

I think it's worth squashing this into the next patch -- without seeing how the harness is used, it's pretty hard to review in isolation.


-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 8
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Thu, 09 Apr 2020 05:18:04 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Adar Dembo, Grant Henke, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15671

to look at the new patch set (#8).

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................

KUDU-3078 Refactor Sentry integration tests

This commit refactors the existing Sentry integration tests to prepare
for Ranger integration tests.

It changes HMS and Sentry integration test base classes to test
harnesses that don't inherit from Gtest to simplify inheritance when
using typed tests.

Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
---
M src/kudu/integration-tests/hms_itest-base.cc
M src/kudu/integration-tests/hms_itest-base.h
M src/kudu/integration-tests/master_hms-itest.cc
M src/kudu/integration-tests/master_sentry-itest.cc
M src/kudu/integration-tests/ts_sentry-itest.cc
5 files changed, 500 insertions(+), 215 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/71/15671/8
-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 8
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Attila Bukor has posted comments on this change. ( http://gerrit.cloudera.org:8080/15671 )

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................


Patch Set 5:

(21 comments)

http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/hms_itest-base.h
File src/kudu/integration-tests/hms_itest-base.h:

http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/hms_itest-base.h@28
PS4, Line 28: #include "kudu/mini-cluster/external_mini_cluster.h"
> Why do we need this anymore?
Done


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/hms_itest-base.h@92
PS4, Line 92: 
> warning: the parameter 'hms_opts' is copied for each invocation but only us
Done


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/hms_itest-base.h@93
PS4, Line 93:   hms::HmsClient* hms_client() {
> warning: passing result of std::move() as a const reference argument; no mo
Done


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/hms_itest-base.cc
File src/kudu/integration-tests/hms_itest-base.cc:

http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/hms_itest-base.cc@78
PS4, Line 78: Status HmsITestHarness::CreateKuduTable(const string& database_name,
> warning: method 'CreateKuduTable' can be made static [readability-convert-m
Done


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/hms_itest-base.cc@121
PS4, Line 121:                                        const boost::optional<const string&>& kudu_table_name) {
> warning: the parameter 'kudu_table_name' is copied for each invocation but 
Done


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/hms_itest-base.cc@186
PS4, Line 186:                                  const boost::optional<const string&>& user,
> warning: the parameter 'user' is copied for each invocation but only used a
Done


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/master_hms-itest.cc
File src/kudu/integration-tests/master_hms-itest.cc:

http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/master_hms-itest.cc@94
PS4, Line 94:   }
            : 
            :   Status StopHms() {
            :     return harness_.StopHms(cluster_);
            :   }
> Why do these need to be internal to the harness?
I thought it's best to wrap everything for consistency. Do you think I should remove these?


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/master_hms-itest.cc@114
PS4, Line 114:     return harness_.CreateHmsTable(database_name, table_name, table_type, kudu_table_name);
> warning: the parameter 'kudu_table_name' is copied for each invocation but 
Done


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/master_hms-itest.cc@136
PS4, Line 136:                   const std::string& table_type = hms::HmsClient::kManagedTable) {
> warning: the parameter 'user' is copied for each invocation but only used a
Done


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc
File src/kudu/integration-tests/master_sentry-itest.cc:

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc@414
PS2, Line 414:                             
> Could you try to articulate what you dislike about the approach?
Posted the next patch and added you as a reviewer. The problem with creating the opts your way is that --sentry_privileges_cache_capacity_mb=$0 is always to opts in case of Sentry with a different $0, so I would need to create the whole opts instead of extending the existing opts from the parent.

I was actually referring to my approach that I don't like, not yours. I just didn't have time to figure out a nice way to solve this, I've been chasing bugs and trying to make this whole thing work before branching.


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/master_sentry-itest.cc
File src/kudu/integration-tests/master_sentry-itest.cc:

http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/master_sentry-itest.cc@170
PS4, Line 170:   static Status GetTableLocationsWithTableId(const string& table_name,
> warning: method 'GetTableLocationsWithTableId' can be made static [readabil
Done


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/master_sentry-itest.cc@171
PS4, Line 171:                                              const optional<const string&>& table_id,
> warning: the parameter 'table_id' is copied for each invocation but only us
Done


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/master_sentry-itest.cc@212
PS4, Line 212:                      const shared_ptr<KuduClient>& client) {
> warning: the const qualified parameter 'client' is copied for each invocati
Done


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/master_sentry-itest.cc@220
PS4, Line 220:   static Status IsCreateTableDone(const OperationParams& p,
> warning: method 'IsCreateTableDone' can be made static [readability-convert
Done


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/master_sentry-itest.cc@226
PS4, Line 226:   static Status DropTable(const OperationParams& p,
> warning: method 'DropTable' can be made static [readability-convert-member-
Done


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/master_sentry-itest.cc@231
PS4, Line 231:   static Status AlterTable(const OperationParams& p,
> warning: method 'AlterTable' can be made static [readability-convert-member
Done


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/master_sentry-itest.cc@238
PS4, Line 238:   static Status IsAlterTableDone(const OperationParams& p,
> warning: method 'IsAlterTableDone' can be made static [readability-convert-
Done


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/master_sentry-itest.cc@244
PS4, Line 244:   static Status RenameTable(const OperationParams& p,
> warning: method 'RenameTable' can be made static [readability-convert-membe
Done


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/master_sentry-itest.cc@251
PS4, Line 251:   static Status GetTableSchema(const OperationParams& p,
> warning: method 'GetTableSchema' can be made static [readability-convert-me
Done


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/master_sentry-itest.cc@262
PS4, Line 262:   static Status GetTabletLocations(const OperationParams& p,
> warning: method 'GetTabletLocations' can be made static [readability-conver
Done


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/master_sentry-itest.cc@296
PS4, Line 296:         ExternalMiniClusterOptions)>& cluster_start_func) {
> warning: the parameter 'cluster_start_func' is copied for each invocation b
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 5
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Wed, 08 Apr 2020 00:34:14 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Adar Dembo, Grant Henke, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15671

to look at the new patch set (#6).

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................

KUDU-3078 Refactor Sentry integration tests

This commit refactors the existing Sentry integration tests to prepare
for Ranger integration tests.

It changes HMS and Sentry integration test base classes to test
harnesses that don't inherit from Gtest to simplify inheritance when
using typed tests.

Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
---
M src/kudu/integration-tests/hms_itest-base.cc
M src/kudu/integration-tests/hms_itest-base.h
M src/kudu/integration-tests/master_hms-itest.cc
M src/kudu/integration-tests/master_sentry-itest.cc
M src/kudu/integration-tests/ts_sentry-itest.cc
5 files changed, 500 insertions(+), 215 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/71/15671/6
-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 6
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Adar Dembo, Grant Henke, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15671

to look at the new patch set (#4).

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................

KUDU-3078 Refactor Sentry integration tests

This commit refactors the existing Sentry integration tests to prepare
for Ranger integration tests.

It changes HMS and Sentry integration test base classes to test
harnesses that don't inherit from Gtest to simplify inheritance when
using typed tests.

Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
---
M src/kudu/integration-tests/hms_itest-base.cc
M src/kudu/integration-tests/hms_itest-base.h
M src/kudu/integration-tests/master_hms-itest.cc
M src/kudu/integration-tests/master_sentry-itest.cc
M src/kudu/integration-tests/ts_sentry-itest.cc
5 files changed, 488 insertions(+), 210 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/71/15671/4
-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 4
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Attila Bukor has abandoned this change. ( http://gerrit.cloudera.org:8080/15671 )

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................


Abandoned

squashed into another commit
-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 8
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Attila Bukor has posted comments on this change. ( http://gerrit.cloudera.org:8080/15671 )

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc
File src/kudu/integration-tests/master_sentry-itest.cc:

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc@371
PS2, Line 371: class RangerITestHarness
> Should we move this to something like master_ranger-itest? Or you will do i
I'm renaming this file to master_authz-itest in a follow-up patch. I think Master and Ranger tests should be in the same file as they will be converted to typed tests.



-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 07 Apr 2020 17:44:43 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Attila Bukor has posted comments on this change. ( http://gerrit.cloudera.org:8080/15671 )

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................


Patch Set 8:

> Patch Set 8:
> 
> I think it's worth squashing this into the next patch -- without seeing how the harness is used, it's pretty hard to review in isolation.

makes sense, squashed it


-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 8
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Thu, 09 Apr 2020 16:11:34 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15671 )

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................


Patch Set 4:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/hms_itest-base.h
File src/kudu/integration-tests/hms_itest-base.h:

http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/hms_itest-base.h@28
PS4, Line 28: #include "kudu/integration-tests/external_mini_cluster-itest-base.h" // IWYU pragma: keep
Why do we need this anymore?


http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/master_hms-itest.cc
File src/kudu/integration-tests/master_hms-itest.cc:

http://gerrit.cloudera.org:8080/#/c/15671/4/src/kudu/integration-tests/master_hms-itest.cc@94
PS4, Line 94:     return harness_.StartHms(cluster_);
            :   }
            : 
            :   Status StopHms() {
            :     return harness_.StopHms(cluster_);
Why do these need to be internal to the harness?


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc
File src/kudu/integration-tests/master_sentry-itest.cc:

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc@414
PS2, Line 414:                             
> Hm I'm not sure it would actually due to how this whole thing is templated 
Could you try to articulate what you dislike about the approach?

Maybe post a WIP of your later patches that actually parameterize to use the harness. I'm finding it really difficult to understand how this harness will be used exactly, and that's making it difficult to understand why everything in this patch is needed.

Taking a step back, I thought the goal of the harnesses was to let us rejigger existing Sentry tests to be Sentry/Ranger agnostic. In doing so, we could then templatize the tests with different harnesses. If that's true, it's surprising that the harnesses would have calls to StartHms/StopHms and StartSentry/StopSentry, given they are decidedly not Sentry/Ranger-agnostic.



-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 4
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Wed, 08 Apr 2020 00:01:22 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Adar Dembo, Grant Henke, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15671

to look at the new patch set (#7).

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................

KUDU-3078 Refactor Sentry integration tests

This commit refactors the existing Sentry integration tests to prepare
for Ranger integration tests.

It changes HMS and Sentry integration test base classes to test
harnesses that don't inherit from Gtest to simplify inheritance when
using typed tests.

Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
---
M src/kudu/integration-tests/hms_itest-base.cc
M src/kudu/integration-tests/hms_itest-base.h
M src/kudu/integration-tests/master_hms-itest.cc
M src/kudu/integration-tests/master_sentry-itest.cc
M src/kudu/integration-tests/ts_sentry-itest.cc
5 files changed, 500 insertions(+), 215 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/71/15671/7
-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 7
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Attila Bukor has posted comments on this change. ( http://gerrit.cloudera.org:8080/15671 )

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................


Patch Set 7:

(15 comments)

http://gerrit.cloudera.org:8080/#/c/15671/6/src/kudu/integration-tests/master_hms-itest.cc
File src/kudu/integration-tests/master_hms-itest.cc:

http://gerrit.cloudera.org:8080/#/c/15671/6/src/kudu/integration-tests/master_hms-itest.cc@107
PS6, Line 107:     return HmsITestHarness::CreateKuduTable(database_name, table_name, client_, timeout);
> warning: static member accessed through instance [readability-static-access
Done


http://gerrit.cloudera.org:8080/#/c/15671/6/src/kudu/integration-tests/master_sentry-itest.cc
File src/kudu/integration-tests/master_sentry-itest.cc:

http://gerrit.cloudera.org:8080/#/c/15671/6/src/kudu/integration-tests/master_sentry-itest.cc@211
PS6, Line 211:   static Status CreateTable(const OperationParams& p,
> warning: method 'CreateTable' can be made static [readability-convert-membe
Done


http://gerrit.cloudera.org:8080/#/c/15671/6/src/kudu/integration-tests/master_sentry-itest.cc@257
PS6, Line 257:   static Status GetTableLocations(const OperationParams& p,
> warning: method 'GetTableLocations' can be made static [readability-convert
Done


http://gerrit.cloudera.org:8080/#/c/15671/6/src/kudu/integration-tests/ts_sentry-itest.cc
File src/kudu/integration-tests/ts_sentry-itest.cc:

http://gerrit.cloudera.org:8080/#/c/15671/6/src/kudu/integration-tests/ts_sentry-itest.cc@249
PS6, Line 249: constexpr int kNumUsers = 3;
> warning: 'kNumUsers' is a static definition in anonymous namespace; static 
Done


http://gerrit.cloudera.org:8080/#/c/15671/6/src/kudu/integration-tests/ts_sentry-itest.cc@250
PS6, Line 250: constexpr const char* kAdminGroup = "admin";
> warning: 'kAdminGroup' is a static definition in anonymous namespace; stati
Done


http://gerrit.cloudera.org:8080/#/c/15671/6/src/kudu/integration-tests/ts_sentry-itest.cc@252
PS6, Line 252: constexpr int kNumTables = 3;
> warning: 'kNumTables' is a static definition in anonymous namespace; static
Done


http://gerrit.cloudera.org:8080/#/c/15671/6/src/kudu/integration-tests/ts_sentry-itest.cc@253
PS6, Line 253: constexpr int kNumColsPerTable = 3;
> warning: 'kNumColsPerTable' is a static definition in anonymous namespace; 
Done


http://gerrit.cloudera.org:8080/#/c/15671/6/src/kudu/integration-tests/ts_sentry-itest.cc@254
PS6, Line 254: constexpr const char* kDb = "db";
> warning: 'kDb' is a static definition in anonymous namespace; static is red
Done


http://gerrit.cloudera.org:8080/#/c/15671/6/src/kudu/integration-tests/ts_sentry-itest.cc@255
PS6, Line 255: constexpr const char* kTablePrefix = "table";
> warning: 'kTablePrefix' is a static definition in anonymous namespace; stat
Done


http://gerrit.cloudera.org:8080/#/c/15671/6/src/kudu/integration-tests/ts_sentry-itest.cc@256
PS6, Line 256: constexpr const char* kAdminRole = "kudu-admin";
> warning: 'kAdminRole' is a static definition in anonymous namespace; static
Done


http://gerrit.cloudera.org:8080/#/c/15671/6/src/kudu/integration-tests/ts_sentry-itest.cc@258
PS6, Line 258: constexpr int kAuthzTokenTTLSecs = 1;
> warning: 'kAuthzTokenTTLSecs' is a static definition in anonymous namespace
Done


http://gerrit.cloudera.org:8080/#/c/15671/6/src/kudu/integration-tests/ts_sentry-itest.cc@259
PS6, Line 259: constexpr int kAuthzCacheTTLMultiplier = 3;
> warning: 'kAuthzCacheTTLMultiplier' is a static definition in anonymous nam
Done


http://gerrit.cloudera.org:8080/#/c/15671/6/src/kudu/integration-tests/ts_sentry-itest.cc@269
PS6, Line 269:         ExternalMiniClusterOptions)>& cluster_start_func) {
> warning: the parameter 'cluster_start_func' is copied for each invocation b
Done


http://gerrit.cloudera.org:8080/#/c/15671/6/src/kudu/integration-tests/ts_sentry-itest.cc@325
PS6, Line 325:   Status CreateTable(const string& table_ident, const shared_ptr<KuduClient>& client) {
> warning: the const qualified parameter 'client' is copied for each invocati
Done


http://gerrit.cloudera.org:8080/#/c/15671/6/src/kudu/integration-tests/ts_sentry-itest.cc@360
PS6, Line 360:     harness_.SetUp([&] (const ExternalMiniClusterOptions& opts) ->
> warning: the parameter 'opts' is copied for each invocation but only used a
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 7
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Wed, 08 Apr 2020 20:05:24 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Attila Bukor has posted comments on this change. ( http://gerrit.cloudera.org:8080/15671 )

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................


Patch Set 8:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/15671/7/src/kudu/integration-tests/ts_sentry-itest.cc
File src/kudu/integration-tests/ts_sentry-itest.cc:

http://gerrit.cloudera.org:8080/#/c/15671/7/src/kudu/integration-tests/ts_sentry-itest.cc@362
PS7, Line 362:                      StartClusterWithOpts(opts);
> warning: std::move of the const variable 'opts' has no effect; remove std::
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 8
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Wed, 08 Apr 2020 21:49:15 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Attila Bukor has posted comments on this change. ( http://gerrit.cloudera.org:8080/15671 )

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................


Patch Set 3:

(9 comments)

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/hms_itest-base.h
File src/kudu/integration-tests/hms_itest-base.h:

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/hms_itest-base.h@36
PS2, Line 36: onst std::unique_ptr<cluster
> client::sp::shared_ptr<> is a shared_ptr template we expose for clients tha
Done


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/hms_itest-base.h@87
PS2, Line 87: 
> Does this need to be public?
Done


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/hms_itest-base.cc
File src/kudu/integration-tests/hms_itest-base.cc:

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/hms_itest-base.cc@59
PS2, Line 59: Status HmsITestHarness::StopHms(const unique_ptr<cluster::ExternalMiniCluster>& cluster) {
> nit: If this is why you're using shared_ptr everywhere, you can pass unique
Done


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_hms-itest.cc
File src/kudu/integration-tests/master_hms-itest.cc:

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_hms-itest.cc@80
PS2, Line 80:     hms_opts.enable_kerberos = EnableKerberos();
            :     hms_opts.service_principal = "hive";
            :     harness_.hms_client_.reset(new HmsC
> nit: maybe add a getter for this. Also, rather than having the client be a 
hm maybe I'm missing something, but how would I reset hms_client_ if I access it via a getter?


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc
File src/kudu/integration-tests/master_sentry-itest.cc:

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc@292
PS2, Line 292: e fun
> nit: maybe rename this to SetupCluster or somesuch? Just so readers don't c
Done


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc@293
PS2, Line 293: tialize the ExternalMiniCluster with the passed
> nit: Should doc what this is expected to do, and maybe note that this might
Done


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc@382
PS2, Line 382: 
> nit: Why were these necessary?
Done


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc@414
PS2, Line 414:                             
> nit: it might be simpler to just define another harness that's basically id
maybe I misunderstood what you're suggesting, but wouldn't that be duplicating the whole setup? That way we would need to make sure to update two separate methods if we need to change anything in the setup code wouldn't we?


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc@413
PS2, Line 413:                             AddExtraOpts(&opts);
             :                             StartClusterWithOpts(opts);
             :                             return cluster_;
             :                           });
             : 
> nit: spacing https://google.github.io/styleguide/cppguide.html#Formatting_L
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 3
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 07 Apr 2020 20:40:48 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Andrew Wong, Adar Dembo, Grant Henke, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15671

to look at the new patch set (#2).

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................

KUDU-3078 Refactor Sentry integration tests

This commit refactors the existing Sentry integration tests to prepare
for Ranger integration tests.

It changes HMS and Sentry integration test base classes to test
harnesses that don't inherit from Gtest to simplify inheritance when
using typed tests.

Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
---
M src/kudu/integration-tests/external_mini_cluster-itest-base.h
M src/kudu/integration-tests/hms_itest-base.cc
M src/kudu/integration-tests/hms_itest-base.h
M src/kudu/integration-tests/master_hms-itest.cc
M src/kudu/integration-tests/master_sentry-itest.cc
M src/kudu/integration-tests/ts_sentry-itest.cc
6 files changed, 456 insertions(+), 190 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/71/15671/2
-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Hao Hao (Code Review)" <ge...@cloudera.org>.
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/15671 )

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................


Patch Set 2: Code-Review+1

(1 comment)

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc
File src/kudu/integration-tests/master_sentry-itest.cc:

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc@371
PS2, Line 371: class RangerITestHarness
Should we move this to something like master_ranger-itest? Or you will do it in a follow up patch?



-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 07 Apr 2020 17:43:24 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Adar Dembo, Grant Henke, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15671

to look at the new patch set (#3).

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................

KUDU-3078 Refactor Sentry integration tests

This commit refactors the existing Sentry integration tests to prepare
for Ranger integration tests.

It changes HMS and Sentry integration test base classes to test
harnesses that don't inherit from Gtest to simplify inheritance when
using typed tests.

Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
---
M src/kudu/integration-tests/hms_itest-base.cc
M src/kudu/integration-tests/hms_itest-base.h
M src/kudu/integration-tests/master_hms-itest.cc
M src/kudu/integration-tests/master_sentry-itest.cc
M src/kudu/integration-tests/ts_sentry-itest.cc
5 files changed, 461 insertions(+), 192 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/71/15671/3
-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 3
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Adar Dembo, Grant Henke, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15671

to look at the new patch set (#5).

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................

KUDU-3078 Refactor Sentry integration tests

This commit refactors the existing Sentry integration tests to prepare
for Ranger integration tests.

It changes HMS and Sentry integration test base classes to test
harnesses that don't inherit from Gtest to simplify inheritance when
using typed tests.

Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
---
M src/kudu/integration-tests/hms_itest-base.cc
M src/kudu/integration-tests/hms_itest-base.h
M src/kudu/integration-tests/master_hms-itest.cc
M src/kudu/integration-tests/master_sentry-itest.cc
M src/kudu/integration-tests/ts_sentry-itest.cc
5 files changed, 498 insertions(+), 215 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/71/15671/5
-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 5
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Attila Bukor has posted comments on this change. ( http://gerrit.cloudera.org:8080/15671 )

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................


Patch Set 3:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_hms-itest.cc
File src/kudu/integration-tests/master_hms-itest.cc:

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_hms-itest.cc@80
PS2, Line 80:     hms_opts.enable_kerberos = EnableKerberos();
            :     hms_opts.service_principal = "hive";
            :     harness_.hms_client_.reset(new HmsC
> It probably wouldn't hurt to move this into the harness itself with some Hm
Done


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc
File src/kudu/integration-tests/master_sentry-itest.cc:

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc@414
PS2, Line 414:                             
> Would something like this work?
Hm I'm not sure it would actually due to how this whole thing is templated and composed now. I also don't really like this approach, but couldn't really find a better way that works. I can try to figure something out later, but I'm trying to get this done before branching.



-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 3
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 07 Apr 2020 23:13:43 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3078 Refactor Sentry integration tests

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15671 )

Change subject: KUDU-3078 Refactor Sentry integration tests
......................................................................


Patch Set 2:

(9 comments)

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/hms_itest-base.h
File src/kudu/integration-tests/hms_itest-base.h:

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/hms_itest-base.h@36
PS2, Line 36: const client::sp::shared_ptr
client::sp::shared_ptr<> is a shared_ptr template we expose for clients that don't have std::shared_ptr. We generally prefer std::shared_ptr where we can.


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/hms_itest-base.h@87
PS2, Line 87:   client::sp::shared_ptr<hms::HmsClient> hms_client_;
Does this need to be public?


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/hms_itest-base.cc
File src/kudu/integration-tests/hms_itest-base.cc:

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/hms_itest-base.cc@59
PS2, Line 59: Status HmsITestHarness::StopHms(const shared_ptr<cluster::ExternalMiniCluster>& cluster) {
nit: If this is why you're using shared_ptr everywhere, you can pass unique pointers as const ref too. Or raw pointers.


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_hms-itest.cc
File src/kudu/integration-tests/master_hms-itest.cc:

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_hms-itest.cc@80
PS2, Line 80:     harness_.hms_client_.reset(new HmsClient(cluster_->hms()->address(), hms_opts));
            :     ASSERT_OK(harness_.hms_client_->Start());
            :     hms_client_ = harness_.hms_client_;
nit: maybe add a getter for this. Also, rather than having the client be a member of the harness _and_ the test, maybe just have it be a member of the harness and call s/hms_client_/harness_.hms_client() ? I think that'd make the lifecycle and ownership of the members easier to follow.


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc
File src/kudu/integration-tests/master_sentry-itest.cc:

http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc@292
PS2, Line 292: SetUp
nit: maybe rename this to SetupCluster or somesuch? Just so readers don't conflate it with the SetUp() function in tests.


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc@293
PS2, Line 293: ExternalMiniClusterOptions)> cluster_start_func
nit: Should doc what this is expected to do, and maybe note that this might return an initialized test member cluster.


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc@382
PS2, Line 382:     opts.extra_master_flags.emplace_back("--trusted_user_acl=impala");
nit: Why were these necessary?


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc@414
PS2, Line 414:         AddExtraOpts(&opts);
nit: it might be simpler to just define another harness that's basically identical to SentryITestHarness, but that has its own GetClusterOpts() that adds the extra opts. That'd be cleaner in that all of the cluster-initialization and specification would be in the harness classes, instead of blurred into the test setup.


http://gerrit.cloudera.org:8080/#/c/15671/2/src/kudu/integration-tests/master_sentry-itest.cc@413
PS2, Line 413:     harness_.SetUp([&] (ExternalMiniClusterOptions opts) -> shared_ptr<ExternalMiniCluster> {
             :         AddExtraOpts(&opts);
             :         StartClusterWithOpts(opts);
             :         return cluster_;
             :         });
nit: spacing https://google.github.io/styleguide/cppguide.html#Formatting_Lambda_Expressions



-- 
To view, visit http://gerrit.cloudera.org:8080/15671
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieff25eaf7a41cf7d8fe965738907e67bbd9b9051
Gerrit-Change-Number: 15671
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 07 Apr 2020 18:49:53 +0000
Gerrit-HasComments: Yes