You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Alexey Serbin (Code Review)" <ge...@cloudera.org> on 2019/05/07 02:49:18 UTC
[kudu-CR] [catalog manager] run sanity checks before catching up with HMS
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13257
Change subject: [catalog_manager] run sanity checks before catching up with HMS
......................................................................
[catalog_manager] run sanity checks before catching up with HMS
When serving CreateTable request, run basic validation and authz-related
checks before waiting for HMS notification listener to catch up.
That's to fail fast if the incoming request doesn't pass basic sanity
verification or isn't authorized.
Change-Id: Ia407554fcd703d424180218f12f23556f54eefeb
---
M src/kudu/master/catalog_manager.cc
1 file changed, 7 insertions(+), 7 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/57/13257/1
--
To view, visit http://gerrit.cloudera.org:8080/13257
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia407554fcd703d424180218f12f23556f54eefeb
Gerrit-Change-Number: 13257
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
[kudu-CR] [catalog manager] run sanity checks before catching up with HMS
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Andrew Wong, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13257
to look at the new patch set (#2).
Change subject: [catalog_manager] run sanity checks before catching up with HMS
......................................................................
[catalog_manager] run sanity checks before catching up with HMS
When serving CreateTable request, run basic validation and authz-related
checks before waiting for HMS notification listener to catch up.
That's to fail fast if the incoming request doesn't pass basic sanity
verification or isn't authorized.
Change-Id: Ia407554fcd703d424180218f12f23556f54eefeb
---
M src/kudu/master/catalog_manager.cc
1 file changed, 6 insertions(+), 6 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/57/13257/2
--
To view, visit http://gerrit.cloudera.org:8080/13257
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia407554fcd703d424180218f12f23556f54eefeb
Gerrit-Change-Number: 13257
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [catalog manager] run sanity checks before catching up with HMS
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/13257 )
Change subject: [catalog_manager] run sanity checks before catching up with HMS
......................................................................
Patch Set 2: Verified+1
known flake: KUDU-2779
--
To view, visit http://gerrit.cloudera.org:8080/13257
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia407554fcd703d424180218f12f23556f54eefeb
Gerrit-Change-Number: 13257
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Comment-Date: Tue, 07 May 2019 05:10:08 +0000
Gerrit-HasComments: No
[kudu-CR] [catalog manager] run sanity checks before catching up with HMS
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/13257 )
Change subject: [catalog_manager] run sanity checks before catching up with HMS
......................................................................
Patch Set 2: Code-Review+2
(1 comment)
http://gerrit.cloudera.org:8080/#/c/13257/2/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
http://gerrit.cloudera.org:8080/#/c/13257/2/src/kudu/master/catalog_manager.cc@1409
PS2, Line 1409: LOG(INFO) << Substitute("Servicing CreateTable request from $0:\n$1",
: RequestorString(rpc), SecureDebugString(req));
One thing worth pointing out: by delaying the WaitForNotificationLogListenerCatchUp call, the timestamp in this log message won't really reflect the time at which the table was actually created. Do we care?
--
To view, visit http://gerrit.cloudera.org:8080/13257
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia407554fcd703d424180218f12f23556f54eefeb
Gerrit-Change-Number: 13257
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Comment-Date: Tue, 07 May 2019 05:09:32 +0000
Gerrit-HasComments: Yes
[kudu-CR] [catalog manager] run sanity checks before catching up with HMS
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/13257 )
Change subject: [catalog_manager] run sanity checks before catching up with HMS
......................................................................
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/13257/1/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
http://gerrit.cloudera.org:8080/#/c/13257/1/src/kudu/master/catalog_manager.cc@1410
PS1, Line 1410: RETURN_NOT_OK(CheckOnline());
> The earlier placement is more consistent with other DDLs.
Done
http://gerrit.cloudera.org:8080/#/c/13257/1/src/kudu/master/catalog_manager.cc@1436
PS1, Line 1436: RETURN_NOT_OK(WaitForNotificationLogListenerCatchUp(resp, rpc));
> Shouldn't we do this before we authorize, in case the privileges change aft
As I understand, granting privileges in Sentry is not reflected in HMS event. For any other case I don't see why it would be important: if that were crucial, I would expect to get some regression failures from our set of tests.
--
To view, visit http://gerrit.cloudera.org:8080/13257
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia407554fcd703d424180218f12f23556f54eefeb
Gerrit-Change-Number: 13257
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 07 May 2019 04:24:31 +0000
Gerrit-HasComments: Yes
[kudu-CR] [catalog manager] run sanity checks before catching up with HMS
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/13257 )
Change subject: [catalog_manager] run sanity checks before catching up with HMS
......................................................................
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/13257/1/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
http://gerrit.cloudera.org:8080/#/c/13257/1/src/kudu/master/catalog_manager.cc@1410
PS1, Line 1410: RETURN_NOT_OK(CheckOnline());
The earlier placement is more consistent with other DDLs.
http://gerrit.cloudera.org:8080/#/c/13257/1/src/kudu/master/catalog_manager.cc@1436
PS1, Line 1436: RETURN_NOT_OK(WaitForNotificationLogListenerCatchUp(resp, rpc));
Shouldn't we do this before we authorize, in case the privileges change after the wait is over? It doesn't matter that much either way I guess.
--
To view, visit http://gerrit.cloudera.org:8080/13257
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia407554fcd703d424180218f12f23556f54eefeb
Gerrit-Change-Number: 13257
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 07 May 2019 03:59:13 +0000
Gerrit-HasComments: Yes
[kudu-CR] [catalog manager] run sanity checks before catching up with HMS
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has removed Kudu Jenkins from this change. ( http://gerrit.cloudera.org:8080/13257 )
Change subject: [catalog_manager] run sanity checks before catching up with HMS
......................................................................
Removed reviewer Kudu Jenkins with the following votes:
* Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/13257
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: Ia407554fcd703d424180218f12f23556f54eefeb
Gerrit-Change-Number: 13257
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
[kudu-CR] [catalog manager] run sanity checks before catching up with HMS
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13257 )
Change subject: [catalog_manager] run sanity checks before catching up with HMS
......................................................................
[catalog_manager] run sanity checks before catching up with HMS
When serving CreateTable request, run basic validation and authz-related
checks before waiting for HMS notification listener to catch up.
That's to fail fast if the incoming request doesn't pass basic sanity
verification or isn't authorized.
Change-Id: Ia407554fcd703d424180218f12f23556f54eefeb
Reviewed-on: http://gerrit.cloudera.org:8080/13257
Reviewed-by: Adar Dembo <ad...@cloudera.com>
Tested-by: Alexey Serbin <as...@cloudera.com>
---
M src/kudu/master/catalog_manager.cc
1 file changed, 6 insertions(+), 6 deletions(-)
Approvals:
Adar Dembo: Looks good to me, approved
Alexey Serbin: Verified
--
To view, visit http://gerrit.cloudera.org:8080/13257
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia407554fcd703d424180218f12f23556f54eefeb
Gerrit-Change-Number: 13257
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
[kudu-CR] [catalog manager] run sanity checks before catching up with HMS
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/13257 )
Change subject: [catalog_manager] run sanity checks before catching up with HMS
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/13257/1/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
http://gerrit.cloudera.org:8080/#/c/13257/1/src/kudu/master/catalog_manager.cc@1436
PS1, Line 1436: RETURN_NOT_OK(WaitForNotificationLogListenerCatchUp(resp, rpc));
> Yeah, I think privileges changes due to rename/drop is caught up at Sentry
Thanks for the confirmation!
--
To view, visit http://gerrit.cloudera.org:8080/13257
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia407554fcd703d424180218f12f23556f54eefeb
Gerrit-Change-Number: 13257
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Comment-Date: Tue, 07 May 2019 19:20:41 +0000
Gerrit-HasComments: Yes
[kudu-CR] [catalog manager] run sanity checks before catching up with HMS
Posted by "Hao Hao (Code Review)" <ge...@cloudera.org>.
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/13257 )
Change subject: [catalog_manager] run sanity checks before catching up with HMS
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/13257/1/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
http://gerrit.cloudera.org:8080/#/c/13257/1/src/kudu/master/catalog_manager.cc@1436
PS1, Line 1436: RETURN_NOT_OK(WaitForNotificationLogListenerCatchUp(resp, rpc));
> As I understand, granting privileges in Sentry is not reflected in HMS even
Yeah, I think privileges changes due to rename/drop is caught up at Sentry server instead of here. And it is hard to coordinate between these two. So it should be fine.
--
To view, visit http://gerrit.cloudera.org:8080/13257
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia407554fcd703d424180218f12f23556f54eefeb
Gerrit-Change-Number: 13257
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Comment-Date: Tue, 07 May 2019 17:57:29 +0000
Gerrit-HasComments: Yes
[kudu-CR] [catalog manager] run sanity checks before catching up with HMS
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/13257 )
Change subject: [catalog_manager] run sanity checks before catching up with HMS
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/13257/2/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
http://gerrit.cloudera.org:8080/#/c/13257/2/src/kudu/master/catalog_manager.cc@1409
PS2, Line 1409: LOG(INFO) << Substitute("Servicing CreateTable request from $0:\n$1",
: RequestorString(rpc), SecureDebugString(req));
> One thing worth pointing out: by delaying the WaitForNotificationLogListene
That's a good observation.
It seems we don't care about that too much: at least in DeleteTableRpc() there is the same story. For aggravated cases I think we need add TRACE() (in separate patch).
--
To view, visit http://gerrit.cloudera.org:8080/13257
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia407554fcd703d424180218f12f23556f54eefeb
Gerrit-Change-Number: 13257
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Comment-Date: Tue, 07 May 2019 05:16:32 +0000
Gerrit-HasComments: Yes