You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/03/10 23:57:04 UTC
[jira] [Commented] (DRILL-5344) External sort priority queue copier
fails with an empty batch
[ https://issues.apache.org/jira/browse/DRILL-5344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15905896#comment-15905896 ]
ASF GitHub Bot commented on DRILL-5344:
---------------------------------------
GitHub user paul-rogers opened a pull request:
https://github.com/apache/drill/pull/778
DRILL-5344: Priority queue copier fails with an empty batch
DRILL-5344: External sort priority queue copier fails with an empty
batch
Unit tests showed that the “priority queue copier” does not handle an
empty batch. This has not been an issue because code elsewhere in the
sort specifically works around this issue. This fix resolves the issue
at the source to avoid the need for future work-arounds.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/paul-rogers/drill DRILL-5344
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/778.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #778
----
commit 24a3bdceaaa975fb5ff9646998fddba5efa6c022
Author: Paul Rogers <pr...@maprtech.com>
Date: 2017-03-10T23:56:18Z
DRILL-5344: Priority queue copier fails with an empty batch
DRILL-5344: External sort priority queue copier fails with an empty
batch
Unit tests showed that the “priority queue copier” does not handle an
empty batch. This has not been an issue because code elsewhere in the
sort specifically works around this issue. This fix resolves the issue
at the source to avoid the need for future work-arounds.
----
> External sort priority queue copier fails with an empty batch
> -------------------------------------------------------------
>
> Key: DRILL-5344
> URL: https://issues.apache.org/jira/browse/DRILL-5344
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.10.0
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Priority: Minor
> Fix For: 1.11.0
>
>
> The external sort uses a "priority queue copier" to merge batches when spilling or when merging spilled batches.
> The code will fail with an {{IndexOutOfBoundsException}} if any record batch is empty. The reason is a faulty assumption in generated code:
> {code}
> public void setup(...) {
> ...
> vector4.set(i, i, batchGroups.get(i).getNextIndex());
> ...
> }
> public int getNextIndex() {
> if (pointer == getRecordCount()) {
> return -1;
> }
> ...
> }
> {code}
> The code to get the next index returns -1 when the "position" in a record batch is zero. The -1 position translates (when truncated) into 65535 which produces the index exception.
> The workaround has been to special case empty batches elsewhere in the code, apparently to avoid hitting this error.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)