[jira] [Commented] (HIVE-1721) use bloom filters to improve the performance of joins

["J. Andrew Key (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-1721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13058125#comment-13058125 ] 

J. Andrew Key commented on HIVE-1721:
-------------------------------------

Is anyone actively working on this?  I've worked with Bloom filters before and was wondering if this issue was perhaps abandoned.  If anyone has any notes or code for me to review, I would love to take a crack at this one.

> use bloom filters to improve the performance of joins
> -----------------------------------------------------
>
>                 Key: HIVE-1721
>                 URL: https://issues.apache.org/jira/browse/HIVE-1721
>             Project: Hive
>          Issue Type: New Feature
>          Components: Query Processor
>            Reporter: Namit Jain
>            Assignee: Siying Dong
>              Labels: optimization
>
> In case of map-joins, it is likely that the big table will not find many matching rows from the small table.
> Currently, we perform a hash-map lookup for every row in the big table, which can be pretty expensive.
> It might be useful to try out a bloom-filter containing all the elements in the small table.
> Each element from the big table is first searched in the bloom filter, and only in case of a positive match,
> the small table hash table is explored.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira