[GitHub] [hive] jcamachor edited a comment on issue #952: HIVE-23006 ProbeDecode compiler support

[GitBox <gi...@apache.org>]

jcamachor edited a comment on issue #952: HIVE-23006 ProbeDecode compiler support
URL: https://github.com/apache/hive/pull/952#issuecomment-608201286
 
 
   @pgaref , thanks for changes. Patch looks good. About the name, I think it is fine (probe side for the filtering generated statically in case of expressions or dynamically for joins).
   
   I think main remaining issue is related to selection in case of multiple MJ operators. I was thinking that for the time being, we could make the policy pluggable via config and have two very simple ones: 1) keep the one with lowest `ndv_JOIN_key_column / ndv_TS_target_column` ratio, or 2) keep those with ratio below a certain threshold specified through a config value. Alternatively, in this patch you could maybe go only with option 1 and tackle other in follow-up.
   
   You have the information about the stats in the operators themselves (follow `op.getStatistics()` calls to see how it is retrieved per column basis). Thus, I would make that decision in the TezCompiler (once we gather all the context information) rather than GenTezUtils. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org