[GitHub] [carbondata] kumarvishal09 commented on pull request #3887: [CARBONDATA-3830] Support Array and Struct of all primitive type reading from presto

[GitBox <gi...@apache.org>]

kumarvishal09 commented on pull request #3887:
URL: https://github.com/apache/carbondata/pull/3887#issuecomment-680904506


   @ajantha-bhat By default Row filter push down is false, In case of complex type for eg: array of string number of records can be much more than 32k, so filling in one shot can be a problem because creating a big array will be an overhead. 
   
   + In case of direct fill we have ResuableDataBuffer to create one big byte array/column for one blocklet processing  and reuse for all the page for a column inside, this wont be useful because number of element in case of complex type can vary, and byte array size will change every page and even creating a big byte array for one page when number of element in child is more can degrade query performance. 
   
   So for complex type user must configure table page size to get the better query performance while creating the table.
   
   If number of row is page is higher we can  avoid ResuableDataBuffer it may be become overhead. 
   
   Some comments:
   1. Pls refactor the code, avoid duplicate code if you can 
   2. Pls add comments.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org