[jira] [Assigned] (BEAM-10379) Some problems when implementing BIT_AND aggregation function in BeamSQL

["Robin Qiu (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-10379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robin Qiu reassigned BEAM-10379:
--------------------------------

    Assignee: Rehman Murad Ali

> Some problems when implementing BIT_AND aggregation function in BeamSQL
> -----------------------------------------------------------------------
>
>                 Key: BEAM-10379
>                 URL: https://issues.apache.org/jira/browse/BEAM-10379
>             Project: Beam
>          Issue Type: Task
>          Components: dsl-sql, dsl-sql-zetasql
>            Reporter: Yuwei Fu
>            Assignee: Rehman Murad Ali
>            Priority: P2
>              Labels: starter
>
> Performs a bitwise AND operation on expression and returns the result.
> Supported Argument Types: INT64
> Returned Data Types: INT64
> Examples
> {code:sql}
> SELECT BIT_AND(c) as bit_and FROM UNNEST([0xF001, 0x00A1]) as c;
> +---------+
> | bit_and |
> +---------+
> | 1       |
> +---------+
> {code}
>  
> What is expected: should include both Calcite and ZetaSQL dialects.
> How to test: unit tests
> Reference: [https://cloud.google.com/bigquery/docs/reference/standard-sql/aggregate_functions#bit_and]
>  
> Problems: 
> After implementation, the current situation is:
>  1. When table is empty, the result is empty, correct.
>  2. When table contains only null values, the result is null, correct.
>  3.  When table contains only valid numerical values, the result is as expected, correct.
>  4. When table contains both valid numerical values and null values, the result should be null (BitAnd(null, val) = null). But it seems all null values have been directly ignored before doing the bit_and operation. Only numerical values are taken into operation, so the result is incorrect.
>  
> It turns out that on direct runner, NULL will not be passed to CombineFn thus all NULL inputs are ignored. And then if there is any non-null inputs, bit_and will be applied on them only, which leads to a non-null result, which is not correct.
> Before figuring out the root cause and proposing a fix, the previous implementation of BIT_AND is reverted.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)