[jira] [Commented] (DRILL-2395) Improve interpreted expression evaluation to only call the setup method once per batch

["Jason Altekruse (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14349766#comment-14349766 ] 

Jason Altekruse commented on DRILL-2395:
----------------------------------------

There was another discussion on the same review about the impact of now using reflection to pass inputs and outputs between expressions. This could also be included with this optimization work. I have copied the comments for this suggestion here as well, we can decide if we want to break this into separate JIRAs when we need to look at improving the interpreter.

Jinfeng:
I assume that we are aware that there is performance overhead to use reflection here. So, in case if we want to use interpreter to evaluate many batches, there would be performance impact. On the other hand, we may argue that using interpreter already means performance is not as good as the current run-time compiling + evaluation model, and therefore, adding additional performance overhead does not change a lot.

Jason:
While there are no instances of evaluating a full batch of records, much less a series of batches in the current code with the interpreter, I am not too concerned about this. If we need a little more perforamnce we could modify the DrillFuncHolderExpr class to hold hard references to the Fields for the inputs extracted out of the DrillFuncCalsses with reflection once during a setup step. We would still be using some reflection to set the values, but we can remove the overhead of finding the fields by name each time. Overall if we need performance for lots of repeatitive expression evaluations, we already have a mechanism for it. I will admit we could probably improve the situation for a medium number of evaluations (maybe in the 10,000 range) with these improvements.

> Improve interpreted expression evaluation to only call the setup method once per batch
> --------------------------------------------------------------------------------------
>
>                 Key: DRILL-2395
>                 URL: https://issues.apache.org/jira/browse/DRILL-2395
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Functions - Drill
>            Reporter: Jason Altekruse
>            Assignee: Daniel Barclay (Drill)
>            Priority: Minor
>
> This enhancement request came out of the review for a patch for DRILL-2060. Copied below is the discussion from there:
> Jinfeng:
> Do you have a plan to move setup() call into places such that setup() will be called once for each VectorAccessible input?
> In the code compile + evaluation model, doSetup() will be called per batch, in stead of per row.
> Jason:
> I have started working on a fix for this. Its a little complicated with setting constant inputs before the setup method is called. I'm trying to figure out the best way to share code with the rest of the input passing used in the EvalVisitor. Would you be okay with this being opened as an enhancement request to be merged a little later? Considering the current use of the interpreter this won't have an impact on any actual queries today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)