[jira] [Created] (DRILL-1601) Ensure a minimum allocation for variable length vectors and improve error messages

["Aman Sinha (JIRA)" <ji...@apache.org>]

Aman Sinha created DRILL-1601:
---------------------------------

             Summary: Ensure a minimum allocation for variable length vectors and improve error messages
                 Key: DRILL-1601
                 URL: https://issues.apache.org/jira/browse/DRILL-1601
             Project: Apache Drill
          Issue Type: Bug
          Components: Execution - Flow
    Affects Versions: 0.6.0
            Reporter: Aman Sinha
            Assignee: Aman Sinha


The allocateNew() and allocateNewSafe() methods for value vectors compute the estimated allocation size based on previous requests.   There could be situations where the allocator determines a very small allocation size based on previous pattern (e.g a bunch of null values) and then encounters a large enough record size.  In such cases, we see the following types of exception: 
  The Partitioner sometimes encounters the following exception:
java.io.IOException: Record {} is too big to fit into the allocated memory of ValueVector.

This issue is mainly applicable for variable length vectors since it uses bytes for the estimation whereas the fixed length vectors use the number of values for estimation. 

A second part of this JIRA is to improve error messages in the Partitioner and UnlimitedRawBatchBuffer. 
        



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)