[jira] [Updated] (DRILL-2088) BitVector.Mutator.setValueCount() fails with invalid buffer access

["Venki Korukanti (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/DRILL-2088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Venki Korukanti updated DRILL-2088:
-----------------------------------
    Attachment: 0001-DRILL-2088-ReAlloc-in-FixedWidthVector.setValueCount.patch

Attaching patch to fix the following issues:
1. In BitVector, find the valueCapacity from the buffer size (similar to other fixed width vectors), not based on the allocation size requested. Currently we store the valueCapacity in a member variable which is not transferred, causing failures when setValueCount is called on vector that got the buffer transfer.

2. Change the FixedLengthVector.java template to generate setValueCount that reallocs when requested valueCount falls beyond the available buffer capacity. This is a problem for nullable vectors (which internally use fixed width vectors). 

Patch has tests for both changes.

> BitVector.Mutator.setValueCount() fails with invalid buffer access
> ------------------------------------------------------------------
>
>                 Key: DRILL-2088
>                 URL: https://issues.apache.org/jira/browse/DRILL-2088
>             Project: Apache Drill
>          Issue Type: Bug
>            Reporter: Venki Korukanti
>            Assignee: Venki Korukanti
>         Attachments: 0001-DRILL-2088-ReAlloc-in-FixedWidthVector.setValueCount.patch
>
>
> Run the following physical plan:
> {code}
> {
>     head:{
>         type:"APACHE_DRILL_PHYSICAL",
>         version:"1",
>         generator:{
>             type:"manual"
>         }
>     },
>     graph:[
>         {
>             @id:1,
>             pop:"mock-scan",
>             url: "http://apache.org",
>             entries:[
>                 {records: 100, types: [
>                   {name: "blue", type: "BIT", mode: "REQUIRED"},
>                   {name: "red", type: "BIGINT", mode: "REQUIRED"},
>                   {name: "green", type: "INT", mode: "OPTIONAL"}
>                 ]}
>             ]
>         },
>         {
>             @id:2,
>             child: 1,
>             pop:"selection-vector-remover"
>         },
>         {
>             @id: 3,
>             child: 2,
>             pop: "screen"
>         }
>     ]
> }
> {code}
> It fails with:
> {code}
> Failure while running fragment., index: 0, length: 8192 (expected: range(0, 1024))
> {code}
> Couple of issues:
> 1. We don't transfer valueCapacity as part of TransferPair.transfer(). This causes the vector that got the buffers to have 0 as valueCapacity. During the realloc of vector that got the buffer, we rely on valueCapacity to decide whether to realloc or not.
> 2. When BitVector.Mutator.setValueCount() is called we try to reallocate the buffer as the valueCapacity is zero. As part of realloc, we allocate a buffer of size 1028bytes and try to copy the existing buffer. Problem is existing buffer has size more than 1028bytes and we fail in copying. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)