You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Igor Guzenko (JIRA)" <ji...@apache.org> on 2019/03/11 12:30:00 UTC
[jira] [Commented] (DRILL-7087) Integrate Arrow's Gandiva into
Drill
[ https://issues.apache.org/jira/browse/DRILL-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16789486#comment-16789486 ]
Igor Guzenko commented on DRILL-7087:
-------------------------------------
Hello [~weijie]. As well as I understand, related changes won't bring fundamental changes of Drill's value vectors internal structure. Looks like you're going to pass our vectors along with computations into Apache Arrow, so then Arrow will perform computation on existing data inside vector. Correct ?
> Integrate Arrow's Gandiva into Drill
> ------------------------------------
>
> Key: DRILL-7087
> URL: https://issues.apache.org/jira/browse/DRILL-7087
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Codegen, Execution - Relational Operators
> Reporter: weijie.tong
> Assignee: weijie.tong
> Priority: Major
>
> It's a prior work to integrate arrow into drill by invoking the its gandiva feature. Comparing arrow and drill 's in memory column representation , there's different null representation internal now. Drill use 1 byte while arrow using 1 bit to indicate one null row. Also all columns of arrow is nullable now. Apart from those basic differences , they have same memory representation to the different data types.
> The integrating strategy is to invoke arrow's JniWrapper's native method directly by passing the ValueVector's memory address.
> I have done a implementation at our own Drill version by integrating gandiva into Drill's project operator. The performance shows that there's nearly 1 times performance gain at expression computation.
> So if there's no objection , I will submit a related PR to contribute this feature. Also this issue waits for arrow's related issue[ARROW-4819].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)