[jira] [Reopened] (KUDU-2305) Local variables can overflow when serializing a 2GB message

["Michael Ho (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KUDU-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Ho reopened KUDU-2305:
------------------------------

> Local variables can overflow when serializing a 2GB message
> -----------------------------------------------------------
>
>                 Key: KUDU-2305
>                 URL: https://issues.apache.org/jira/browse/KUDU-2305
>             Project: Kudu
>          Issue Type: Bug
>          Components: rpc
>    Affects Versions: 1.6.0
>            Reporter: Joe McDonnell
>            Assignee: Joe McDonnell
>            Priority: Major
>             Fix For: 1.7.0
>
>
> When rpc_max_message_size is set to its maximum of INT_MAX (2147483647), certain local variables in SerializeMessage can overflow as messages approach this size. Specifically, recorded_size, size_with_delim, and total_size are 4 byte signed integers and could overflow when additional_size becomes large.
> Since INT_MAX is the largest allowable value for rpc_max_message_size (a 4 byte signed integer), these variables will not overflow if changed to 4 byte unsigned integers. This would eliminate the potential problem for serialization.
> A similar problem exists in the InboundTransfer::ReceiveBuffer() and similar codepaths. Changing those variables to unsigned integers should resolve the issue.
> This does not impact existing systems, because the default value of rpc_max_message_size is 50MB.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)