[jira] [Created] (DRILL-5828) RecordBatchLoader permutes column order

["Paul Rogers (JIRA)" <ji...@apache.org>]

Paul Rogers created DRILL-5828:
----------------------------------

             Summary: RecordBatchLoader permutes column order
                 Key: DRILL-5828
                 URL: https://issues.apache.org/jira/browse/DRILL-5828
             Project: Apache Drill
          Issue Type: Bug
    Affects Versions: 1.11.0
            Reporter: Paul Rogers
            Priority: Minor


The {{RecordBatchLoader}} class deserializes batches and checks for schema changes. As part of investigating DRILL-5826, it seems that {{RecordBatchLoader}} detects schema changes as follows:

* If two batches have the same column in the same order, no schema change occurs. (Fine)
* If batch A has schema (a, b) while batch B has (b, a), then no schema change occurs. (Fine)

But, in the case of permutated columns (second case above), the {{RecordBatchLoader}} returns the column order of the second batch, though it says that no schema change has occurred.

That is, {{RecordBatchLoader}} says that the schema has not changed, but the actual schema has changed (column order changed.)

This is a potential problem: if a downstream batch counts on the same column order, then that assumption is violated by the behavior described above.

Correct behavior would be to coerce the second batch to match the schema of the first batch, if the {{RecordBatchLoader}} indicates that no schema change occurred.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)