[jira] [Commented] (IMPALA-11141) Use exact data types in IN-list filter

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-11141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528551#comment-17528551 ] 

ASF subversion and git services commented on IMPALA-11141:
----------------------------------------------------------

Commit 6380a3187c2d0efeadfa7c8a4ff5746b0e16e6ab in impala's branch refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=6380a3187 ]

IMPALA-11141: Use exact data types in IN-list filter

Currently, we use a std::unordered_set<int64_t> for all numeric types
(including DATE type). It's a waste of space for small data types like
tinyint, smallint, int, etc. This patch extends the base InListFilter
class with native implementations for different data types.

For string type in-list filters, this patch uses impala::StringValue
instead of std::string. This simplifies the Insert() method, which
improves the codegen time. To use impala::StringValue, this patch
switches the set implementation to boost::unordered_set. Same as what we
use in InPredicate.

Another improvement of using impala::StringValue is that we can easily
maintain the strings in MemPool. When inserting a new batch of values,
the new values are inserted into a temp set. String pointers still
reference to the original tuple values. At the end of processing each
batch, MaterializeValues() is invoked to copy the strings into the
filter's own mem pool. This is more memory-friendly than the original
approach since we can allocate the string batch at once.

Tests:
 - Add unit tests for different types of in-list filters

Change-Id: Id434a542b2ced64efa3bfc974cb565b94a4193e9
Reviewed-on: http://gerrit.cloudera.org:8080/18433
Reviewed-by: Qifan Chen <qc...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Use exact data types in IN-list filter
> --------------------------------------
>
>                 Key: IMPALA-11141
>                 URL: https://issues.apache.org/jira/browse/IMPALA-11141
>             Project: IMPALA
>          Issue Type: Improvement
>            Reporter: Quanlong Huang
>            Assignee: Quanlong Huang
>            Priority: Major
>
> Currently, we use a std::unordered_set<int64_t> for all integer types (including DATE type). It's a waste of space for small data types like tinyint, smallint, int, etc. We'd better extends the base InListFilter class with different implementations for different data types.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org