[GitHub] [drill] Z0ltrix commented on pull request #2412: DRILL-8088: Improve expression evaluation performance

[GitBox <gi...@apache.org>]

Z0ltrix commented on pull request #2412:
URL: https://github.com/apache/drill/pull/2412#issuecomment-1003945110


   > > Drill was designed to allow vector operations (hence Value Vectors), but the code was never written. In part because there are no CPU vector instructions that work with SQL nullable data. Arrow is supposed to have figured out solutions (Gandiva, is it?) which, perhaps we could consider (but probably only for non-nullable data.)
   > 
   > Hi @paul-rogers, I think that what Arrow does for computations over nullable data is store an external null mask and compute results for every record, including the null ones where the value vector contains either rubbish or some sentinel value. In a second pass, a null mask is computed for the result. This results in wasted arithmetic operations for null values, but in practice that's better than a branch for every value. Quite possibly even for pretty sparse vectors.
   
   i'm sure you have already discussed this, but i would like to know why we are not migrating to arrow and cannot find any information about this decision. As far as i know, arrow was inspired by drill and on the arrow homepage they have still the picture with drill on it but drill does not use arrow. https://arrow.apache.org/overview/ 
   Is there any official statement from the project for the arrow support/migration?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org