You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Jacques Nadeau (JIRA)" <ji...@apache.org> on 2015/10/07 18:15:27 UTC
[jira] [Created] (DRILL-3907) Enable ScanBatch to provide a merge
on pre-sorted readers
Jacques Nadeau created DRILL-3907:
-------------------------------------
Summary: Enable ScanBatch to provide a merge on pre-sorted readers
Key: DRILL-3907
URL: https://issues.apache.org/jira/browse/DRILL-3907
Project: Apache Drill
Issue Type: Bug
Components: Storage - Other
Reporter: Jacques Nadeau
Fix For: 1.3.0
In some situations, individual record readers will be presorted by a key. Given Drill's approach to parallelization, it is possible that a Single ScanBatch will be interacting with multiple readers. If we want to maintain the collation of the underlying data, we need to Drill to do a n-way merge on the streams as they are read into Drill. This functionality already exists in the MergingReceiver.
This JIRA is to refactor merging receiver so that the underlying N-Way merge of batches can be used in other locations. We then need to decide whether to incorporate it directly into the ScanBatch (when needed) or to do something external. We also need to resolve how we decide whether the collation that could be provided by utilizing an n-way merge is necessary (to avoid paying the cost of maintaining an unused collation).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)