You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Beam JIRA Bot (Jira)" <ji...@apache.org> on 2020/10/06 17:10:00 UTC
[jira] [Updated] (BEAM-10379) Some problems when implementing
BIT_AND aggregation function in BeamSQL
[ https://issues.apache.org/jira/browse/BEAM-10379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Beam JIRA Bot updated BEAM-10379:
---------------------------------
Labels: stale-assigned starter (was: starter)
> Some problems when implementing BIT_AND aggregation function in BeamSQL
> -----------------------------------------------------------------------
>
> Key: BEAM-10379
> URL: https://issues.apache.org/jira/browse/BEAM-10379
> Project: Beam
> Issue Type: Task
> Components: dsl-sql, dsl-sql-zetasql
> Reporter: Yuwei Fu
> Assignee: Shehzaad Nakhoda
> Priority: P2
> Labels: stale-assigned, starter
>
> Performs a bitwise AND operation on expression and returns the result.
> Supported Argument Types: INT64
> Returned Data Types: INT64
> Examples
> {code:sql}
> SELECT BIT_AND(c) as bit_and FROM UNNEST([0xF001, 0x00A1]) as c;
> +---------+
> | bit_and |
> +---------+
> | 1 |
> +---------+
> {code}
>
> What is expected: should include both Calcite and ZetaSQL dialects.
> How to test: unit tests
> Reference: [https://cloud.google.com/bigquery/docs/reference/standard-sql/aggregate_functions#bit_and]
>
> Problems:
> After implementation, the current situation is:
> 1. When table is empty, the result is empty, correct.
> 2. When table contains only null values, the result is null, correct.
> 3. When table contains only valid numerical values, the result is as expected, correct.
> 4. When table contains both valid numerical values and null values, the result should be null (BitAnd(null, val) = null). But it seems all null values have been directly ignored before doing the bit_and operation. Only numerical values are taken into operation, so the result is incorrect.
>
> It turns out that on direct runner, NULL will not be passed to CombineFn thus all NULL inputs are ignored. And then if there is any non-null inputs, bit_and will be applied on them only, which leads to a non-null result, which is not correct.
> Before figuring out the root cause and proposing a fix, the previous implementation of BIT_AND is reverted.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)