You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@fluss.apache.org by "wuchong (via GitHub)" <gi...@apache.org> on 2026/08/06 07:59:47 UTC

Re: [PR] [docs] Add RoaringBitmap SQL functions documentation [fluss]

wuchong commented on code in PR #3823:
URL: https://github.com/apache/fluss/pull/3823#discussion_r3726880258


##########
website/docs/engine-flink/sql-functions.md:
##########
@@ -0,0 +1,333 @@
+---
+id: sql-functions
+title: SQL Functions
+sidebar_label: SQL Functions
+sidebar_position: 7

Review Comment:
   Both SQL Functions and Delta Joins currently use `sidebar_position: 7`, so this page is not explicitly placed after Delta Joins as stated in the PR description. Please set this value to `8`.



##########
website/docs/engine-flink/sql-functions.md:
##########
@@ -0,0 +1,333 @@
+---
+id: sql-functions
+title: SQL Functions
+sidebar_label: SQL Functions
+sidebar_position: 7
+---
+
+# SQL Functions
+
+Apache Fluss registers a set of built-in SQL functions in `FlussCatalog`. These are **Flink-side
+functions** that execute within the Flink query engine. They are distinct from the storage-level
+`rbm32` / `rbm64` aggregators, which execute inside the Fluss TabletServer during write.
+
+## How to Use
+
+After creating a Fluss catalog and setting it as the active catalog, all functions are available
+in Flink SQL without any `CREATE TEMPORARY FUNCTION` statement.
+
+```sql
+-- 1. Create the catalog (replace connection details as needed)
+CREATE CATALOG fluss_catalog WITH (
+    'type'         = 'fluss',
+    'bootstrap.servers' = 'localhost:9123'
+);
+
+-- 2. Switch to the catalog
+USE CATALOG fluss_catalog;
+
+-- 3. Use any bitmap function directly
+SELECT rb_cardinality(rb_build(ARRAY[1, 2, 3, 2]));
+-- Output: 3
+```
+
+All functions below operate on `BYTES` columns containing standard 32-bit RoaringBitmap
+serialized data — the same wire format used by the `rbm32` storage-level aggregator.
+
+---
+
+## Aggregate Functions

Review Comment:
   Could we place Scalar Functions before Aggregate Functions? `rb_build`, `rb_cardinality`, and the other scalar functions are the foundational operations for creating, inspecting, and manipulating a bitmap, and the introductory example already uses them. Presenting single-row bitmap operations before cross-row aggregation avoids forward references and gives the page a more natural progression. Within the scalar section, please introduce `rb_build` before `rb_cardinality`, since `rb_build` is the primitive used throughout the examples.



##########
website/docs/table-design/merge-engines/aggregation.md:
##########
@@ -970,6 +970,12 @@ TableDescriptor.builder()
 </TabItem>
 </Tabs>
 
+:::tip
+To query and manipulate bitmap columns from Flink SQL, use the built-in RoaringBitmap SQL
+functions (`rb_build_agg`, `rb_or_agg`, `rb_cardinality`, etc.) provided by FlussCatalog.

Review Comment:
   Could we also update the `rbm32` Flink SQL example above to construct bitmap values with `rb_build(ARRAY[...])` instead of opaque `X3A30...` literals? This makes the example easier to understand and lets users modify and rerun it without manually producing serialized bytes. Likewise, please select `rb_to_array(visit_bitmap)` in the result query so the union is shown in a human-readable form, and remove or update the note saying that values must be pre-serialized on the client side.



##########
website/docs/engine-flink/sql-functions.md:
##########
@@ -0,0 +1,333 @@
+---
+id: sql-functions
+title: SQL Functions
+sidebar_label: SQL Functions
+sidebar_position: 7
+---
+
+# SQL Functions
+
+Apache Fluss registers a set of built-in SQL functions in `FlussCatalog`. These are **Flink-side
+functions** that execute within the Flink query engine. They are distinct from the storage-level
+`rbm32` / `rbm64` aggregators, which execute inside the Fluss TabletServer during write.
+
+## How to Use
+
+After creating a Fluss catalog and setting it as the active catalog, all functions are available
+in Flink SQL without any `CREATE TEMPORARY FUNCTION` statement.
+
+```sql
+-- 1. Create the catalog (replace connection details as needed)
+CREATE CATALOG fluss_catalog WITH (
+    'type'         = 'fluss',
+    'bootstrap.servers' = 'localhost:9123'
+);
+
+-- 2. Switch to the catalog
+USE CATALOG fluss_catalog;
+
+-- 3. Use any bitmap function directly
+SELECT rb_cardinality(rb_build(ARRAY[1, 2, 3, 2]));
+-- Output: 3
+```
+
+All functions below operate on `BYTES` columns containing standard 32-bit RoaringBitmap
+serialized data — the same wire format used by the `rbm32` storage-level aggregator.
+
+---
+
+## Aggregate Functions
+
+Aggregate functions reduce multiple rows into a single bitmap result.
+
+### rb_build_agg
+
+Builds a serialized `RoaringBitmap` from a column of `INT` values across rows.
+
+**Signature:** `rb_build_agg(value INT) → BYTES`
+
+| Input | Output |
+|---|---|
+| NULL | ignored |
+| all NULL | `NULL` |
+
+```sql
+-- Input
+-- user_id
+-- 1
+-- 2
+-- 3
+-- 2  (duplicate)
+
+SELECT rb_cardinality(rb_build_agg(user_id)) AS uv
+FROM (VALUES (1), (2), (3), (2)) AS t(user_id);
+
+-- Output: 3
+```
+
+---
+
+### rb_or_agg
+
+Unions multiple serialized `RoaringBitmap` values via bitwise OR across rows.
+
+**Signature:** `rb_or_agg(bitmap BYTES) → BYTES`
+
+| Input | Output |
+|---|---|
+| NULL rows | ignored |
+| all NULL | `NULL` |
+
+```sql
+-- Input: two bitmaps {1,2} and {2,3}
+SELECT rb_cardinality(rb_or_agg(bmap)) AS result
+FROM (
+    VALUES
+        (X'3A30000000000000020000000102'),  -- bitmap {1, 2}

Review Comment:
   Please build these inputs with `rb_build` rather than hard-coded serialized bytes, for example by placing arrays in `VALUES` and applying `rb_build` before `rb_or_agg`. This keeps the example self-explanatory and avoids opaque binary literals whose contents readers cannot verify or safely modify.



##########
website/docs/engine-flink/sql-functions.md:
##########
@@ -0,0 +1,333 @@
+---
+id: sql-functions
+title: SQL Functions
+sidebar_label: SQL Functions
+sidebar_position: 7
+---
+
+# SQL Functions
+
+Apache Fluss registers a set of built-in SQL functions in `FlussCatalog`. These are **Flink-side
+functions** that execute within the Flink query engine. They are distinct from the storage-level
+`rbm32` / `rbm64` aggregators, which execute inside the Fluss TabletServer during write.
+
+## How to Use
+
+After creating a Fluss catalog and setting it as the active catalog, all functions are available
+in Flink SQL without any `CREATE TEMPORARY FUNCTION` statement.
+
+```sql
+-- 1. Create the catalog (replace connection details as needed)
+CREATE CATALOG fluss_catalog WITH (
+    'type'         = 'fluss',
+    'bootstrap.servers' = 'localhost:9123'
+);
+
+-- 2. Switch to the catalog
+USE CATALOG fluss_catalog;
+
+-- 3. Use any bitmap function directly
+SELECT rb_cardinality(rb_build(ARRAY[1, 2, 3, 2]));
+-- Output: 3
+```
+
+All functions below operate on `BYTES` columns containing standard 32-bit RoaringBitmap
+serialized data — the same wire format used by the `rbm32` storage-level aggregator.
+
+---
+
+## Aggregate Functions
+
+Aggregate functions reduce multiple rows into a single bitmap result.
+
+### rb_build_agg
+
+Builds a serialized `RoaringBitmap` from a column of `INT` values across rows.
+
+**Signature:** `rb_build_agg(value INT) → BYTES`
+
+| Input | Output |

Review Comment:
   The null-handling tables take up a large portion of the page and feel repetitive, while the same rules would be easier to scan as short prose. Please use the earlier compact format and add a `**Null Handling:** ...` item alongside `**Signature:**` for each function instead of using a table.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org