You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2023/03/21 17:47:00 UTC
[jira] [Work logged] (HIVE-27113) Increasing default for hive.thrift.client.max.message.size to 2 GB
[ https://issues.apache.org/jira/browse/HIVE-27113?focusedWorklogId=852073&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-852073 ]
ASF GitHub Bot logged work on HIVE-27113:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 21/Mar/23 17:46
Start Date: 21/Mar/23 17:46
Worklog Time Spent: 10m
Work Description: rtrivedi12 opened a new pull request, #4137:
URL: https://github.com/apache/hive/pull/4137
…e to 2 GB
### What changes were proposed in this pull request?
Changed the default value for thrift message size to 2147483647 bytes instead of 1 GB and fixed config help message.
Also, Wrapped the config value to a max of INTEGER.MAX_VALUE.
### Why are the changes needed?
Wide tables with huge partitions (5k+) can cross current thrift max message size of 1GB as the partition object contains column descriptors and other properties set by Impala.
Help message suggested the upper limit for max message size as 2 GB which is outside of INTEGER.MAX_VALUE range and hence caused overflow when converted to int and got updated to thrift default (100 MB)
### How was this patch tested?
There is no functionality change introduced, so existing test cases should not be failing with these config value changes
Issue Time Tracking
-------------------
Worklog Id: (was: 852073)
Remaining Estimate: 0h
Time Spent: 10m
> Increasing default for hive.thrift.client.max.message.size to 2 GB
> ------------------------------------------------------------------
>
> Key: HIVE-27113
> URL: https://issues.apache.org/jira/browse/HIVE-27113
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Reporter: Riju Trivedi
> Assignee: Riju Trivedi
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> HIVE_THRIFT_CLIENT_MAX_MESSAGE_SIZE("hive.thrift.client.max.message.size", "1gb",
> new SizeValidator(-1L, true, (long) Integer.MAX_VALUE, true),
> "Thrift client configuration for max message size. 0 or -1 will use the default defined in the Thrift " +
> "library. The upper limit is 2147483648 bytes (or 2gb).")
> Documentation on the help suggests setting 2147483648 while Integer Max is 2147483647. So, it actually becomes -1 and gets set to thrift default limit (100 MB)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)