[jira] [Commented] (DRILL-6048) ListVector is incomplete and broken, RepeatedListVector works

["Paul Rogers (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16300525#comment-16300525 ] 

Paul Rogers commented on DRILL-6048:
------------------------------------

Correct conversion from non-union to union type can be found in {{SchemaUtil.coerceVector()}}.

> ListVector is incomplete and broken, RepeatedListVector works
> -------------------------------------------------------------
>
>                 Key: DRILL-6048
>                 URL: https://issues.apache.org/jira/browse/DRILL-6048
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.10.0
>            Reporter: Paul Rogers
>
> Drill provides two kinds of "list vectors": {{ListVector}} and {{RepeatedListVector}}. I attempted to use the {{ListVector}} to implement lists in JSON. While some parts work, others are broken and JIRA tickets were filed.
> Once things worked well enough to run a query, it turned out that the Project operator failed. Digging into the cause, it appears that the {{ListVector}} is incomplete and not used. Its implementation of {{makeTransferPair()}} was clearly never tested. A list has contents, but when this method attempts to create the contents of the target vector, it fails to create the list contents.
> Elsewhere, we saw that the constructor did correctly create the vector, and that the {{promoteToUnion()}} had holes. The sheer number of bugs leads to the conclusion that this class is not, in fact, used or usable.
> Looking more carefully at the JSON and older writer code, it appears that the ListVector was *not* used for JSON, and that JSON has the limitations of a repeated vector (it cannot support lists with null elements.)
> This implies that the JSON reader itself is broken as it does not support fully JSON semantics because it does not use the {{ListVector}} that was intended for this purpose.
> So, the conclusion is that JSON uses:
> * Repeated vectors for single-dimensional arrays (without null support)
> * {{RepeatedListVector}} for two-dimensional arrays
> This triggers the question: what do we do for three-dimensional arrays?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)