You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by "Ajantha Bhat (JIRA)" <ji...@apache.org> on 2018/09/11 16:00:00 UTC
[jira] [Created] (CARBONDATA-2927) Multiple issue fixes for varchar
column and complex columns that grows more than 2MB
Ajantha Bhat created CARBONDATA-2927:
----------------------------------------
Summary: Multiple issue fixes for varchar column and complex columns that grows more than 2MB
Key: CARBONDATA-2927
URL: https://issues.apache.org/jira/browse/CARBONDATA-2927
Project: CarbonData
Issue Type: Bug
Reporter: Ajantha Bhat
Assignee: Ajantha Bhat
*Fixed:*
*1. varchar data length is more than 2MB, buffer overflow exception (thread local row buffer)*
*root* casue*: thread* loaclbuffer *was hardcoded with 2MB.*
*solution: grow dynamically based on the row size.*
*2. read data from carbon file having one row of varchar data with 150 MB length is very slow.*
*root casue: At UnsafeDMStore, ensure memory is just incresing by 8KB each time and lot of time malloc and free happens before reaching 150MB. hence very slow performance.*
*solution: directly check and allocate the required size.*
*3. Jvm crash when data size is more than 128 MB in unsafe sort step.*
*root cause: unsafeCarbonRowPage is of 128MB, so if data is more than 128MB for one row, we access block beyond allocated, leading to JVM crash.*
*solution: validate the size before access and prompt user to increase unsafe memory. (by carbon property)*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)