[jira] [Resolved] (IMPALA-4874) Increase maximum KRPC message size

["Joe McDonnell (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-4874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joe McDonnell resolved IMPALA-4874.
-----------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 2.12.0

commit baec8cae34591c92c9309333280f85f81d852f8d
Author: Joe McDonnell <jo...@cloudera.com>
Date: Thu Feb 15 13:10:21 2018 -0800

IMPALA-4874: Increase maximum KRPC message size
 
 The default value for rpc_max_message_size is 50MB.
 Impala currently requires support for messages of
 up to 2GB. This changes the value of rpc_max_message_size
 to INT_MAX for Impala.
 
 Testing:
 - Added a test to test_very_large_strings that generates
 a row with multiple large strings. This row requires
 that the RPC framework successfully transmit over
 400MB. This works for both KRPC and Thrift.
 This query operates under the same amount of memory
 as other queries in large_strings.test.
 - Tested separately that larger row sizes also work,
 including tests up to almost 2GB.
 
 Change-Id: I876bba0536e1d85e41eacd9c0aeccfe5c2126e58
 Reviewed-on: http://gerrit.cloudera.org:8080/9337
 Reviewed-by: Joe McDonnell <jo...@cloudera.com>
 Tested-by: Impala Public Jenkins

> Increase maximum KRPC message size
> ----------------------------------
>
>                 Key: IMPALA-4874
>                 URL: https://issues.apache.org/jira/browse/IMPALA-4874
>             Project: IMPALA
>          Issue Type: Sub-task
>          Components: Distributed Exec
>    Affects Versions: Impala 2.9.0
>            Reporter: Henry Robinson
>            Assignee: Joe McDonnell
>            Priority: Major
>             Fix For: Impala 2.12.0
>
>
> KRPC has a natural maximum message size (controlled by {{--rpc_max_message_size}}), but also uses Protobuf in such a way as the maximum message size is separately restricted to 512MB.
> Although Impala should try and avoid any payloads of this size, realistically we may not get rid of all of them by Impala 2.9.0. For now, let's increase the max message size while at the same time working to reduce the actual payload size created by Impala.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)