[jira] [Created] (CARBONDATA-3761) Remove redundant conversion for complex type insert

["Ajantha Bhat (Jira)" <ji...@apache.org>]

Ajantha Bhat created CARBONDATA-3761:
----------------------------------------

             Summary: Remove redundant conversion for complex type insert
                 Key: CARBONDATA-3761
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-3761
             Project: CarbonData
          Issue Type: Improvement
            Reporter: Ajantha Bhat
            Assignee: Ajantha Bhat


h3. Problem
 # In {{PrimitiveDataType#writeByteArray}}
{{DataTypeUtil.parseValue(**input.toString()**, carbonDimension)}}
Here we convert every complex child element to string and then parse as an object to handle bad records. Which leads to heavy GC

 # {{DatatypeUtil#getBytesDataDataTypeForNoDictionaryColumn}} -> double,float, byte, decimal case is missing. so we convert them to string and then convert to bytes. which create more redundant objects

h3. Solution
 # For new Insert into flow, no need to handle bad records for complex types as it is already validated in source table. So, use object directly. This can decrease the memory foot print for complex type insert
 # Add a case for double,float, byte, decimal data type.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)