[kudu-CR] KUDU-3450 handling of oversized messages in subprocess server

["Alexey Serbin (Code Review)" <ge...@cloudera.org>]

Hello Ashwani Raina, Attila Bukor, Kudu Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19500

to look at the new patch set (#2).

Change subject: KUDU-3450 handling of oversized messages in subprocess server
......................................................................

KUDU-3450 handling of oversized messages in subprocess server

This patch adds a new flag --subprocess_max_message_size_bytes to
configure the maximum allowed size for the body of the response message
received by the subprocess server, with default value of 8 MiB.
Prior to this patch, the hard-coded limit was 1 MiB.

I also updated the default values for the following flags, increasing
them 4x times:
  * --subprocess_request_queue_size_bytes       (4 --> 16 MiB)
  * --subprocess_response_queue_size_bytes      (4 --> 16 MiB)

With this patch, the behavior of the subprocess server has changed
when encountering an error while reading a response message from
its subprocess.  Now, server does not just bail upon a larger than
expected message or in case of other error, but rather tries to read
out and discard the message to clear the communication channel,
so there is a chance to receive next messages from the subprocess.

A new metric has been added for the subprocess server to report on
the number of read and discarded messages because of various errors
while reading and decoding the data from the communication channel:
  * server_dropped_messages

In addition, this patch introduces a few test scenarios to cover the
newly introduced functionality.

I also sprayed the relevant code with syntactic sugar of structural
binding since the code requires C++17-capable compiler anyways.

A follow-up patch should take care of the corresponding client-side
components that are used to run the Ranger client as a subprocess.

Change-Id: I05b09e757f304b22e37438c2445ecc161ef412c9
---
M src/kudu/ranger/ranger_client.cc
M src/kudu/subprocess/echo_subprocess.cc
M src/kudu/subprocess/server.cc
M src/kudu/subprocess/server.h
M src/kudu/subprocess/subprocess_protocol.cc
M src/kudu/subprocess/subprocess_protocol.h
M src/kudu/subprocess/subprocess_proxy-test.cc
M src/kudu/subprocess/subprocess_server-test.cc
8 files changed, 311 insertions(+), 68 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/00/19500/2
-- 
To view, visit http://gerrit.cloudera.org:8080/19500
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I05b09e757f304b22e37438c2445ecc161ef412c9
Gerrit-Change-Number: 19500
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)