[GitHub] [incubator-doris] kangkaisen opened a new pull request #1610: Add bitmap agg type and udaf

[GitBox <gi...@apache.org>]

kangkaisen opened a new pull request #1610: Add bitmap agg type and udaf
URL: https://github.com/apache/incubator-doris/pull/1610
 
 
   for https://github.com/apache/incubator-doris/issues/1486 and https://github.com/apache/incubator-doris/issues/1485
   
   **The change for this PR:**
   
   1 Add a bitmap_union agg type
   
   2 Add a bitmap udaf
   
   3 Add a bitmap_inin function, convert a int value to bitmap
   
   4 Remove aligned_free method in port.h, because which is conflict with RoaringBitmap portability.h.
   
   5 Remove merge method from aggregate_func, becuase all agg input is aggregated intermediate data. So the merge method and update method we only need one.
   
   6 Remove unused Field constructor method
   
   7 Use the Field polymorphism eliminate if-else.
   
   8 why change the AggregateInfo::init api?
   
   because key column and none agg value column no update semantic，so we need fill the dst with src by init method
   
   
   **How to use:**
   
   1 creata table with bitmap_union:
   
   ```
   CREATE TABLE `bitmap_test` (
     `id` int(11) NULL COMMENT "",
     `id2` varchar(100) bitmap_count NULL
   ) ENGINE=OLAP
   AGGREGATE KEY(`id`)
   DISTRIBUTED BY HASH(`id`) BUCKETS 10;
   ```
   
   2 load the data by stream load:
   
   ```
   seq 10 13 | awk '{OFS="\t"}{print $1, $1 * 10}' | curl --location-trusted -u xxx:xxx  -T - http://hostname:8410/api/test/bitmap_test/_stream_load
   ```
   
   3 query:
   ```
   select bitmap_union(id2) from bitmap_test;
   
   select id, bitmap_union(id2) from bitmap_test group by id ;
   ```
   
   **Todo:**
   
   - [ ] avoid serialize and deserialize from Scannode to Aggnode
   
   - [ ] add a new type for hll_union and bitmap_union
   
   - [ ] bitmap_union support insert into select
   
   - [ ] bitmap_union support broker load
   
   - [ ] add document for bitmap_union
   
   - [ ] add test for bitmap_union
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@doris.apache.org
For additional commands, e-mail: dev-help@doris.apache.org