[jira] [Assigned] (IMPALA-10112) Consider skipping FpRateTooHigh() check for bloom filters

["Riza Suminto (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-10112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Riza Suminto reassigned IMPALA-10112:
-------------------------------------

    Assignee: Riza Suminto  (was: Tim Armstrong)

> Consider skipping FpRateTooHigh() check for bloom filters
> ---------------------------------------------------------
>
>                 Key: IMPALA-10112
>                 URL: https://issues.apache.org/jira/browse/IMPALA-10112
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Backend
>            Reporter: Tim Armstrong
>            Assignee: Riza Suminto
>            Priority: Major
>              Labels: performance
>
> This check disables bloom filters on the sender side.
> It is inaccurate in cases where there are duplicate values of the filter key on the build side. E.g. many-to-many join or a join with multiple keys. This could be fixed with some effort, but is probably not worth it, because:
> * Partition filters are probably still worth evaluating even if there are false positives, because it's cheap and eliminating a partition is still beneficial.
> * Runtime filters are dynamically disabled on the scan side if they are ineffective. I think we still also "evaluate" the always true filter, which is cheaper than doing the hashing and bloom evaluation, but still not entirely free.
> * The disabling is fairly unlikely to kick in for partitioned joins because it's only applied to a small subset of the filter, before the Or() operation.
> So it's potentially harmful and only likely beneficial for broadcast join filters, in which case it saves a small amount of scan CPU and, for global filters, coordinator RPCs and broadcasting. It's unclear that the complexity is worth it for this relatively small and uncertain benefit.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org