[kudu] 02/02: [util] GetBindIpForDaemon() consumes PID wider than 18 bit

[al...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

alexey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git

commit ee86f568cb405f8636ae56f3b69f8009f8f481da
Author: Alexey Serbin <al...@apache.org>
AuthorDate: Sat Mar 21 12:41:33 2020 -0700

    [util] GetBindIpForDaemon() consumes PID wider than 18 bit
    
    This patch updates the logic of GetBindIpForDaemon() to accept process
    identifiers wider than 18 bits.  Prior to this patch, maximum allowable
    PID was 262143, and feeding in a higher PID lead to a crash.  The
    limitation was due to the mapping scheme of (PID, index) --> loopback
    IP address: it mapped 18-bit PID and 6-bit index into the 24-bit space
    of 127.0.0.0/8 loopback addresses.
    
    With this patch, GetBindIpForDaemon() stops imposing the limit on the
    maximum PID.  Now the uniqueness of the generated IP address is not
    guaranteed in general if OS generates PIDs wider than 18 bits, but it
    still holds across all Kudu test processes if both of the following
    conditions are true:
    
      * PIDs are monotonically increasing with a delta D which is much
        smaller than 262143 for newly spawned processes (usually D is 1)
    
      * no more than 262144/D processes are spawned between the earliest
        still running process and the latest running one while executing
        Kudu tests involving external mini-cluster
    
    The former item is not true for hardened Linux distros where PID
    assignment is randomized, but it's unlikely that anybody is interested
    running Kudu tests on such systems.
    
    Essentially, the new scheme uses PID % 262144 as input for the PID part,
    keeping it in the 18-bit space even if the original PID address space is
    wider (it might be up to 22 bits wide for 64-bit Linux distros).
    
    This patch helps running Kudu tests on Docker containers where the host
    OS is configured to have PIDs higher than 262143 and updating maximum
    PID is not an option since /proc/sys/kernel/pid_max is mounted
    read-only.
    
    This patch relates to the following JIRA issues:
      KUDU-1334
      KUDU-3044
    
    Change-Id: Idf2162a2a976a0ce997f02a321f230e7f5878328
    Reviewed-on: http://gerrit.cloudera.org:8080/15519
    Reviewed-by: Adar Dembo <ad...@cloudera.com>
    Tested-by: Kudu Jenkins
---
 src/kudu/util/net/net_util.cc | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/src/kudu/util/net/net_util.cc b/src/kudu/util/net/net_util.cc
index 6599f5c..81835c1 100644
--- a/src/kudu/util/net/net_util.cc
+++ b/src/kudu/util/net/net_util.cc
@@ -492,11 +492,16 @@ string GetBindIpForDaemon(int index, BindMode bind_mode) {
   CHECK(0 < index && index <= kServersMaxNum) << Substitute(
       "server index $0 is not in range ($1, $2]", index, 0, kServersMaxNum);
 
+  static constexpr uint32_t kMaxPid = 1 << kPidBits;
   switch (bind_mode) {
     case BindMode::UNIQUE_LOOPBACK: {
       uint32_t pid = getpid();
-      CHECK_LT(pid, 1 << kPidBits) << Substitute(
-          "PID $0 is more than $1 bits wide", pid, kPidBits);
+      if (pid >= kMaxPid) {
+        LOG(INFO) << Substitute(
+            "PID $0 is more than $1 bits wide, substituted with $2",
+            pid, kPidBits, pid % kMaxPid);
+        pid %= kMaxPid;
+      }
       uint32_t ip = (pid << kServerIdxBits) | static_cast<uint32_t>(index);
       uint8_t octets[] = {
           static_cast<uint8_t>((ip >> 16) & 0xff),