[jira] [Commented] (DRILL-6043) Nullable vector, but not List vector, adds its internal vectors to child list

["Paul Rogers (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-6043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16296436#comment-16296436 ] 

Paul Rogers commented on DRILL-6043:
------------------------------------

Once the list passes through project, it does pick up the internal vectors:

{noformat}
`flat`(LIST:OPTIONAL) [`[DEFAULT]`(LATE:OPTIONAL),
`$data$`(LIST:OPTIONAL) ...]]]
{noformat}

But, the way this information was added is broken. Project added a vector for the data portion of the list. But, it did remove the original "dummy" type created when the list is created. This leaves the list with two children when it should have one.

We must have multiple ways that we manipulate list internals, leading to these errors.

> Nullable vector, but not List vector, adds its internal vectors to child list
> -----------------------------------------------------------------------------
>
>                 Key: DRILL-6043
>                 URL: https://issues.apache.org/jira/browse/DRILL-6043
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.10.0
>            Reporter: Paul Rogers
>            Priority: Minor
>
> Each Drill vector has associated metadata in the form of a {{MaterializeField}} instance. The {{MaterializeField}} contains a list of children. For a Map vector, the list of children lists the vectors that make up the map.
> Nullable vectors use the list of children to identify the hidden vectors that make up the nullable vectors: {{$bits$}} and {{$values$}}.
> However, repeated vectors (including lists) also have hidden internal vectors: offsets and values. However, the metadata for repeated types and lists do not include these in the vector metadata.
> We should decide if we need metadata for the implied internal vectors. (Having it does cause problems since a newly-created schema for a nullable vector is not equal to the actual schema created by the vector itself.)
> If we don't need the internal vector metadata, remove it from the nullable vectors.
> But, if we do need it, add it to the repeated vectors and to lists.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)