You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/08/01 12:43:00 UTC
[jira] [Work logged] (BEAM-5056) [SQL] Nullability of aggregation
expressions isn't inferred properly
[ https://issues.apache.org/jira/browse/BEAM-5056?focusedWorklogId=129671&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129671 ]
ASF GitHub Bot logged work on BEAM-5056:
----------------------------------------
Author: ASF GitHub Bot
Created on: 01/Aug/18 12:42
Start Date: 01/Aug/18 12:42
Worklog Time Spent: 10m
Work Description: kanterov opened a new pull request #6118: [BEAM-5056] [SQL] Fix nullability in output schema
URL: https://github.com/apache/beam/pull/6118
Before we didn't properly propagate nullability from Calcite to
`Schema.Field`.
In the case of "group by", aggregate expressions were always non-nullable.
It isn't true if the bucketing key and aggregated expression are nullable.
```
> SELECT SUM(EXPR$0), EXPR$0 FROM UNNEST (ARRAY [1, NULL]) GROUP BY EXPR$0;
+------------+------------+
| EXPR$0 | EXPR$0 |
+------------+------------+
| null | null |
| 1 | 1 |
+------------+------------+
```
In the case of "join", every field was nullable. It true for outer joins,
but isn't precise for left, right or inner joins.
The code is based on https://github.com/apache/beam/pull/6108. One suggestion would be to check output schema in tests, along with output.
------------------------
Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
It will help us expedite review of your Pull Request if you tag someone (e.g. `@username`) to look at it.
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/) | --- | --- | --- | --- | --- | ---
Java | [](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/) </br> [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | --- | --- | --- | ---
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 129671)
Time Spent: 10m
Remaining Estimate: 0h
> [SQL] Nullability of aggregation expressions isn't inferred properly
> --------------------------------------------------------------------
>
> Key: BEAM-5056
> URL: https://issues.apache.org/jira/browse/BEAM-5056
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql
> Reporter: Gleb Kanterov
> Assignee: Xu Mingmin
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Given schema and rows:
> {code:java}
> Schema schema =
> Schema.builder()
> .addNullableField("f_int1", Schema.FieldType.INT32)
> .addNullableField("f_int2", Schema.FieldType.INT32)
> .build();
> List<Row> rows =
> TestUtils.RowsBuilder.of(schema)
> .addRows(null, null)
> .getRows();
> {code}
> Following query fails:
> {code:sql}
> SELECT AVG(f_int1) FROM PCOLLECTION GROUP BY f_int2
> {code}
> {code:java}
> Caused by: java.lang.IllegalArgumentException: Field EXPR$0 is not nullable{code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)