You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Alexey Serbin (Code Review)" <ge...@cloudera.org> on 2023/02/20 05:37:43 UTC
[kudu-CR] KUDU-3450 handling of oversized messages in Ranger process wrapper
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19516
Change subject: KUDU-3450 handling of oversized messages in Ranger process wrapper
......................................................................
KUDU-3450 handling of oversized messages in Ranger process wrapper
This patch updates the Ranger client process wrapper to handle
oversized messages. With this patch, upon receiving an oversized
message, the wrapper discards the message and logs about the error.
The input stream is ready to receive next messages after that.
A corresponding unit test for MessageIO.readBytes() is added as well.
In addition, now the --subprocess_max_message_size_bytes master's flag
controls the maximum message size at the Ranger client process wrapper
as well. SubprocessConfiguration.MAX_MESSAGE_BYTES_DEFAULT has been
updated to match the default value of the flag.
This is a follow-up to ae22d32ef5895a02a763665cfd2812aa61be5ab3.
Change-Id: I9fe57bdeeb8a5515578e3feb7cacbbb0d7692e3e
---
M java/kudu-subprocess/src/main/java/org/apache/kudu/subprocess/MessageIO.java
M java/kudu-subprocess/src/main/java/org/apache/kudu/subprocess/MessageReader.java
M java/kudu-subprocess/src/main/java/org/apache/kudu/subprocess/SubprocessConfiguration.java
M java/kudu-subprocess/src/test/java/org/apache/kudu/subprocess/TestMessageIO.java
M src/kudu/ranger/ranger_client.cc
5 files changed, 83 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/16/19516/1
--
To view, visit http://gerrit.cloudera.org:8080/19516
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I9fe57bdeeb8a5515578e3feb7cacbbb0d7692e3e
Gerrit-Change-Number: 19516
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
[kudu-CR] KUDU-3450 handling of oversized messages in Ranger process wrapper
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Yingchun Lai, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19516
to look at the new patch set (#3).
Change subject: KUDU-3450 handling of oversized messages in Ranger process wrapper
......................................................................
KUDU-3450 handling of oversized messages in Ranger process wrapper
This patch updates the Ranger client process wrapper to handle
oversized messages. With this patch, upon receiving an oversized
message, the wrapper discards the message and logs about the error.
The input stream is ready to receive next messages after that.
A corresponding unit test for MessageIO.readBytes() is added as well.
In addition, now the --subprocess_max_message_size_bytes master's flag
controls the maximum message size at the Ranger client process wrapper
as well. SubprocessConfiguration.MAX_MESSAGE_BYTES_DEFAULT has been
updated to match the default value of the flag.
This is a follow-up to ae22d32ef5895a02a763665cfd2812aa61be5ab3.
Change-Id: I9fe57bdeeb8a5515578e3feb7cacbbb0d7692e3e
---
M java/kudu-subprocess/src/main/java/org/apache/kudu/subprocess/MessageIO.java
M java/kudu-subprocess/src/main/java/org/apache/kudu/subprocess/MessageReader.java
M java/kudu-subprocess/src/main/java/org/apache/kudu/subprocess/SubprocessConfiguration.java
M java/kudu-subprocess/src/test/java/org/apache/kudu/subprocess/TestMessageIO.java
M src/kudu/ranger/ranger_client.cc
5 files changed, 79 insertions(+), 26 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/16/19516/3
--
To view, visit http://gerrit.cloudera.org:8080/19516
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9fe57bdeeb8a5515578e3feb7cacbbb0d7692e3e
Gerrit-Change-Number: 19516
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai <la...@apache.org>
[kudu-CR] KUDU-3450 handling of oversized messages in Ranger process wrapper
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19516 )
Change subject: KUDU-3450 handling of oversized messages in Ranger process wrapper
......................................................................
KUDU-3450 handling of oversized messages in Ranger process wrapper
This patch updates the Ranger client process wrapper to handle
oversized messages. With this patch, upon receiving an oversized
message, the wrapper discards the message and logs about the error.
The input stream is ready to receive next messages after that.
A corresponding unit test for MessageIO.readBytes() is added as well.
In addition, now the --subprocess_max_message_size_bytes master's flag
controls the maximum message size at the Ranger client process wrapper
as well. SubprocessConfiguration.MAX_MESSAGE_BYTES_DEFAULT has been
updated to match the default value of the flag.
This is a follow-up to ae22d32ef5895a02a763665cfd2812aa61be5ab3.
Change-Id: I9fe57bdeeb8a5515578e3feb7cacbbb0d7692e3e
Reviewed-on: http://gerrit.cloudera.org:8080/19516
Tested-by: Alexey Serbin <al...@apache.org>
Reviewed-by: Attila Bukor <ab...@apache.org>
---
M java/kudu-subprocess/src/main/java/org/apache/kudu/subprocess/MessageIO.java
M java/kudu-subprocess/src/main/java/org/apache/kudu/subprocess/MessageReader.java
M java/kudu-subprocess/src/main/java/org/apache/kudu/subprocess/SubprocessConfiguration.java
M java/kudu-subprocess/src/test/java/org/apache/kudu/subprocess/TestMessageIO.java
M src/kudu/ranger/ranger_client.cc
5 files changed, 79 insertions(+), 26 deletions(-)
Approvals:
Alexey Serbin: Verified
Attila Bukor: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/19516
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I9fe57bdeeb8a5515578e3feb7cacbbb0d7692e3e
Gerrit-Change-Number: 19516
Gerrit-PatchSet: 4
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai <la...@apache.org>
[kudu-CR] KUDU-3450 handling of oversized messages in Ranger process wrapper
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/19516 )
Change subject: KUDU-3450 handling of oversized messages in Ranger process wrapper
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/19516/2/java/kudu-subprocess/src/test/java/org/apache/kudu/subprocess/TestMessageIO.java
File java/kudu-subprocess/src/test/java/org/apache/kudu/subprocess/TestMessageIO.java:
http://gerrit.cloudera.org:8080/#/c/19516/2/java/kudu-subprocess/src/test/java/org/apache/kudu/subprocess/TestMessageIO.java@158
PS2, Line 158: Assert.fail(String.format("unexpected exception: %s", e.getMessage()));
> why not just add throws Exception to the signature?
Indeed!
--
To view, visit http://gerrit.cloudera.org:8080/19516
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9fe57bdeeb8a5515578e3feb7cacbbb0d7692e3e
Gerrit-Change-Number: 19516
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai <la...@apache.org>
Gerrit-Comment-Date: Wed, 01 Mar 2023 02:10:24 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3450 handling of oversized messages in Ranger process wrapper
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/19516 )
Change subject: KUDU-3450 handling of oversized messages in Ranger process wrapper
......................................................................
Patch Set 3: Verified+1
unrelated test failure in TxnOpDispatcherITest.TxnWriteWhileReplicaDeleted
--
To view, visit http://gerrit.cloudera.org:8080/19516
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9fe57bdeeb8a5515578e3feb7cacbbb0d7692e3e
Gerrit-Change-Number: 19516
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai <la...@apache.org>
Gerrit-Comment-Date: Wed, 01 Mar 2023 03:19:18 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3450 handling of oversized messages in Ranger process wrapper
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Yingchun Lai, Attila Bukor, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19516
to look at the new patch set (#2).
Change subject: KUDU-3450 handling of oversized messages in Ranger process wrapper
......................................................................
KUDU-3450 handling of oversized messages in Ranger process wrapper
This patch updates the Ranger client process wrapper to handle
oversized messages. With this patch, upon receiving an oversized
message, the wrapper discards the message and logs about the error.
The input stream is ready to receive next messages after that.
A corresponding unit test for MessageIO.readBytes() is added as well.
In addition, now the --subprocess_max_message_size_bytes master's flag
controls the maximum message size at the Ranger client process wrapper
as well. SubprocessConfiguration.MAX_MESSAGE_BYTES_DEFAULT has been
updated to match the default value of the flag.
This is a follow-up to ae22d32ef5895a02a763665cfd2812aa61be5ab3.
Change-Id: I9fe57bdeeb8a5515578e3feb7cacbbb0d7692e3e
---
M java/kudu-subprocess/src/main/java/org/apache/kudu/subprocess/MessageIO.java
M java/kudu-subprocess/src/main/java/org/apache/kudu/subprocess/MessageReader.java
M java/kudu-subprocess/src/main/java/org/apache/kudu/subprocess/SubprocessConfiguration.java
M java/kudu-subprocess/src/test/java/org/apache/kudu/subprocess/TestMessageIO.java
M src/kudu/ranger/ranger_client.cc
5 files changed, 83 insertions(+), 26 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/16/19516/2
--
To view, visit http://gerrit.cloudera.org:8080/19516
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9fe57bdeeb8a5515578e3feb7cacbbb0d7692e3e
Gerrit-Change-Number: 19516
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai <la...@apache.org>
[kudu-CR] KUDU-3450 handling of oversized messages in Ranger process wrapper
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has removed a vote on this change.
Change subject: KUDU-3450 handling of oversized messages in Ranger process wrapper
......................................................................
Removed Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/19516
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I9fe57bdeeb8a5515578e3feb7cacbbb0d7692e3e
Gerrit-Change-Number: 19516
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai <la...@apache.org>
[kudu-CR] KUDU-3450 handling of oversized messages in Ranger process wrapper
Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Attila Bukor has posted comments on this change. ( http://gerrit.cloudera.org:8080/19516 )
Change subject: KUDU-3450 handling of oversized messages in Ranger process wrapper
......................................................................
Patch Set 3: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/19516
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9fe57bdeeb8a5515578e3feb7cacbbb0d7692e3e
Gerrit-Change-Number: 19516
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai <la...@apache.org>
Gerrit-Comment-Date: Wed, 01 Mar 2023 05:30:29 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3450 handling of oversized messages in Ranger process wrapper
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/19516 )
Change subject: KUDU-3450 handling of oversized messages in Ranger process wrapper
......................................................................
Patch Set 1: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/19516
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9fe57bdeeb8a5515578e3feb7cacbbb0d7692e3e
Gerrit-Change-Number: 19516
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai <la...@apache.org>
Gerrit-Comment-Date: Tue, 21 Feb 2023 15:31:53 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-3450 handling of oversized messages in Ranger process wrapper
Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Attila Bukor has posted comments on this change. ( http://gerrit.cloudera.org:8080/19516 )
Change subject: KUDU-3450 handling of oversized messages in Ranger process wrapper
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/19516/2/java/kudu-subprocess/src/test/java/org/apache/kudu/subprocess/TestMessageIO.java
File java/kudu-subprocess/src/test/java/org/apache/kudu/subprocess/TestMessageIO.java:
http://gerrit.cloudera.org:8080/#/c/19516/2/java/kudu-subprocess/src/test/java/org/apache/kudu/subprocess/TestMessageIO.java@158
PS2, Line 158: Assert.fail(String.format("unexpected exception: %s", e.getMessage()));
why not just add throws Exception to the signature?
--
To view, visit http://gerrit.cloudera.org:8080/19516
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9fe57bdeeb8a5515578e3feb7cacbbb0d7692e3e
Gerrit-Change-Number: 19516
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai <la...@apache.org>
Gerrit-Comment-Date: Tue, 28 Feb 2023 14:36:51 +0000
Gerrit-HasComments: Yes