[jira] [Created] (CARBONDATA-2901) Problem: Jvm crash in Load scenario when unsafe memory allocation is failed.

["Ajantha Bhat (JIRA)" <ji...@apache.org>]

Ajantha Bhat created CARBONDATA-2901:
----------------------------------------

             Summary: Problem: Jvm crash in Load scenario when unsafe memory allocation is failed.
                 Key: CARBONDATA-2901
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2901
             Project: CarbonData
          Issue Type: Bug
            Reporter: Ajantha Bhat
            Assignee: Ajantha Bhat


Problem: Jvm crash in Load scenario when unsafe memory allocation is failed.

scenario: 
a) Have many cores while loading. suggested more than 10. [carbon.number.of.cores.while.loading]
b) Load huge data with local sort, more than 5GB (keeping default unsafe memory manager as 512 MB)
c) when task failes due to not enough unsafae memory, JVM crashes with SIGSEGV.


root casue:
while sorting, all iterator threads are waiting at UnsafeSortDataRows.addRowBatch as all iterator works on one row page.
Only one iterator thread will try to allocate memory. Before that it has freed current page in handlePreviousPage().
When allocate memory failed, row page will still have that old reference. next thread will again use same reference and call handlePreviousPage().
So, Jvm crashes as freed memory is accessed.


solution:
When allocation failed, set row page reference to null.
So, that next thread will not do any operation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)