[GitHub] paul-rogers commented on issue #1414: DRILL-6101: Optimized implicit columns handling within scanner

[GitBox <gi...@apache.org>]

paul-rogers commented on issue #1414: DRILL-6101: Optimized implicit columns handling within scanner
URL: https://github.com/apache/drill/pull/1414#issuecomment-409800290
 
 
   As it turns out, optimized processing for implicit columns is available in the long-stalled result set loader pull requests. That code grabs only the data columns (which it packs optimally into a result set), then adds implicit columns in a later step, avoiding the thrashing that can occur if the implicit columns are populated concurrent with incoming data.
   
   What is really needed is for the planner to recognize the implicit columns and project them separately from the data. That is `SELECT *, filename` should result in projecting `*` from the table, and only the `filename` implicit column. Else, the best we can do is generate the columns, then throw them away, which is obviously suboptimal.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services