You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2021/02/22 18:16:21 UTC
[GitHub] [beam] sonam-vend opened a new pull request #14035: draft implementation of arr_concat_Agg
sonam-vend opened a new pull request #14035:
URL: https://github.com/apache/beam/pull/14035
**Please** add a meaningful description for your change here
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Dataflow | Flink | Samza | Spark | Twister2
--- | --- | --- | --- | --- | --- | ---
Go | [](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) | ---
Java | [](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://ci-beam
.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/)
Python | [](https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/) | ---
XLang | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/) | ---
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website | Whitespace | Typescript
--- | --- | --- | --- | --- | --- | ---
Non-portable | [](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/lastCompletedBuild/) <br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/)
Portable | --- | [](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | --- | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #14035: [BEAM-11849] Support ARRAY_CONCAT_AGG fn for Zetasql dialect
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #14035:
URL: https://github.com/apache/beam/pull/14035#issuecomment-804517420
> > What would be a reasonable next step?
>
> We could merge this PR but with ARRAY_CONCAT_AGG left commented out in the supported functions list. However there would not be much benefit in doing so. Really we need to fix [BEAM-10379](https://issues.apache.org/jira/browse/BEAM-10379), which now blocks many aggregate function implementations from passing compliance tests.
Could we have a plan to fix BEAM-10379? @kennknowles @tysonjh @apilloud
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ibzib commented on pull request #14035: [BEAM-11849] Support ARRAY_CONCAT_AGG fn for Zetasql dialect
Posted by GitBox <gi...@apache.org>.
ibzib commented on pull request #14035:
URL: https://github.com/apache/beam/pull/14035#issuecomment-805034293
> > > What would be a reasonable next step?
> >
> >
> > We could merge this PR but with ARRAY_CONCAT_AGG left commented out in the supported functions list. However there would not be much benefit in doing so. Really we need to fix [BEAM-10379](https://issues.apache.org/jira/browse/BEAM-10379), which now blocks many aggregate function implementations from passing compliance tests.
>
> Could we have a plan to fix [BEAM-10379](https://issues.apache.org/jira/browse/BEAM-10379)? @kennknowles @tysonjh @apilloud
@aaltay let's discuss offline
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] sonam-vend commented on a change in pull request #14035: [BEAM-11849] Support ARRAY_CONCAT_AGG fn for Zetasql dialect
Posted by GitBox <gi...@apache.org>.
sonam-vend commented on a change in pull request #14035:
URL: https://github.com/apache/beam/pull/14035#discussion_r591517959
##########
File path: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlOperators.java
##########
@@ -92,6 +93,14 @@
x -> createTypeFactory().createArrayType(x.getOperandType(0), -1),
new UdafImpl<>(new ArrayAgg.ArrayAggArray()));
+ public static final SqlOperator ARRAY_CONCAT_AGG_FN =
+ createUdafOperator(
+ "array_concat_agg",
+ x ->
+ createTypeFactory()
+ .createArrayType(createTypeFactory().createSqlType(SqlTypeName.BIGINT), -1),
Review comment:
That was really a huge bug you identified. Thanks @ibzib :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] sonam-vend commented on pull request #14035: [BEAM-11849] Support ARRAY_CONCAT_AGG fn for Zetasql dialect
Posted by GitBox <gi...@apache.org>.
sonam-vend commented on pull request #14035:
URL: https://github.com/apache/beam/pull/14035#issuecomment-791905929
R: @robinyqiu @ibzib
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #14035: [BEAM-11849] Support ARRAY_CONCAT_AGG fn for Zetasql dialect
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #14035:
URL: https://github.com/apache/beam/pull/14035#issuecomment-803210310
> Unfortunately, this PR fails compliance tests because of nullness issues. I'm guessing it's nothing wrong with this PR, it's probably an unrelated blocker ([BEAM-10379](https://issues.apache.org/jira/browse/BEAM-10379)).
What would be a reasonable next step?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #14035: [BEAM-11849] Support ARRAY_CONCAT_AGG fn for Zetasql dialect
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #14035:
URL: https://github.com/apache/beam/pull/14035#issuecomment-802335541
Is this ready for another review?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ibzib commented on a change in pull request #14035: [BEAM-11849] Support ARRAY_CONCAT_AGG fn for Zetasql dialect
Posted by GitBox <gi...@apache.org>.
ibzib commented on a change in pull request #14035:
URL: https://github.com/apache/beam/pull/14035#discussion_r583044780
##########
File path: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlOperators.java
##########
@@ -91,6 +92,12 @@
x -> createTypeFactory().createArrayType(x.getOperandType(0), -1),
new UdafImpl<>(new ArrayAgg.ArrayAggArray()));
+ public static final SqlOperator ARRAY_CONCAT_AGG_FN =
+ createUdafOperator(
+ "array_concat_agg",
+ x -> createTypeFactory().createArrayType(x.getOperandType(0), -1),
Review comment:
I'm pretty sure the problem is with this line here. The return type is supposed to be ARRAY<BIGINT>, but the type returned here is ARRAY<ARRAY<BIGINT>>. I'm guessing it's because `x.getOperandType(0)` is an array type. Instead it should be the type of elements inside `x.getOperandType(0)`. (I'm not sure exactly what the method is to get that, but you should be able to figure it out using the debugger.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] closed pull request #14035: [BEAM-11849] Support ARRAY_CONCAT_AGG fn for Zetasql dialect
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #14035:
URL: https://github.com/apache/beam/pull/14035
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #14035: [BEAM-11849] Support ARRAY_CONCAT_AGG fn for Zetasql dialect
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14035:
URL: https://github.com/apache/beam/pull/14035#issuecomment-1037214352
This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ibzib commented on a change in pull request #14035: [BEAM-11849] Support ARRAY_CONCAT_AGG fn for Zetasql dialect
Posted by GitBox <gi...@apache.org>.
ibzib commented on a change in pull request #14035:
URL: https://github.com/apache/beam/pull/14035#discussion_r589663990
##########
File path: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlOperators.java
##########
@@ -92,6 +93,14 @@
x -> createTypeFactory().createArrayType(x.getOperandType(0), -1),
new UdafImpl<>(new ArrayAgg.ArrayAggArray()));
+ public static final SqlOperator ARRAY_CONCAT_AGG_FN =
+ createUdafOperator(
+ "array_concat_agg",
+ x ->
+ createTypeFactory()
+ .createArrayType(createTypeFactory().createSqlType(SqlTypeName.BIGINT), -1),
Review comment:
You need to get the array element type from `x`. Otherwise this function will only work for integers.
##########
File path: sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlDialectSpecTest.java
##########
@@ -3843,4 +3847,27 @@ public void testArrayAggZetasql() {
pipeline.run().waitUntilFinish(Duration.standardMinutes(PIPELINE_EXECUTION_WAITTIME_MINUTES));
}
+
+ @Test
+ public void testArrayConcatAggZetasql() {
+ String sql =
+ "WITH aggregate_example AS (SELECT [1,2] AS numbers UNION ALL SELECT [3,4] AS numbers UNION ALL SELECT [5, 6] AS numbers) SELECT ARRAY_CONCAT_AGG(numbers) AS count_to_six_agg FROM aggregate_example";
+
+ ZetaSQLQueryPlanner zetaSQLQueryPlanner = new ZetaSQLQueryPlanner(config);
+ BeamRelNode beamRelNode = zetaSQLQueryPlanner.convertToBeamRel(sql);
+ PCollection<Row> stream = BeamSqlRelUtils.toPCollection(pipeline, beamRelNode);
+ PAssert.thatSingleton(stream)
+ .satisfies(
+ row -> {
+ Collection<Object> output = row.getArray("count_to_six_agg");
+ HashSet<Object> outputSet = new HashSet<Object>(output);
+
+ HashSet<Object> expectedOutputSet = Sets.newHashSet(1L, 2L, 3L, 4L, 5L, 6L);
+
+ assertThat("array_field", expectedOutputSet.equals(outputSet));
Review comment:
Use `containsInAnyOrder` instead of building and comparing sets. It's simpler. http://hamcrest.org/JavaHamcrest/javadoc/1.3/org/hamcrest/Matchers.html#containsInAnyOrder(T...)
##########
File path: sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlDialectSpecTest.java
##########
@@ -3843,4 +3847,27 @@ public void testArrayAggZetasql() {
pipeline.run().waitUntilFinish(Duration.standardMinutes(PIPELINE_EXECUTION_WAITTIME_MINUTES));
}
+
+ @Test
+ public void testArrayConcatAggZetasql() {
+ String sql =
+ "WITH aggregate_example AS (SELECT [1,2] AS numbers UNION ALL SELECT [3,4] AS numbers UNION ALL SELECT [5, 6] AS numbers) SELECT ARRAY_CONCAT_AGG(numbers) AS count_to_six_agg FROM aggregate_example";
Review comment:
Break this query string into multiple lines.
##########
File path: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlOperators.java
##########
@@ -266,7 +275,7 @@ public SqlSyntax getSyntax() {
};
}
- private static RelDataType createSqlType(SqlTypeName typeName, boolean withNullability) {
+ public static RelDataType createSqlType(SqlTypeName typeName, boolean withNullability) {
Review comment:
Why does this need to be public?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ibzib commented on pull request #14035: [BEAM-11849] Support ARRAY_CONCAT_AGG fn for Zetasql dialect
Posted by GitBox <gi...@apache.org>.
ibzib commented on pull request #14035:
URL: https://github.com/apache/beam/pull/14035#issuecomment-804328404
> What would be a reasonable next step?
We could merge this PR but with ARRAY_CONCAT_AGG left commented out in the supported functions list. However there would not be much benefit in doing so. Really we need to fix BEAM-10379, which now blocks many aggregate function implementations from passing compliance tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] closed pull request #14035: [BEAM-11849] Support ARRAY_CONCAT_AGG fn for Zetasql dialect
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #14035:
URL: https://github.com/apache/beam/pull/14035
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #14035: [BEAM-11849] Support ARRAY_CONCAT_AGG fn for Zetasql dialect
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14035:
URL: https://github.com/apache/beam/pull/14035#issuecomment-1037214352
This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] sonam-vend commented on a change in pull request #14035: [BEAM-11849] Support ARRAY_CONCAT_AGG fn for Zetasql dialect
Posted by GitBox <gi...@apache.org>.
sonam-vend commented on a change in pull request #14035:
URL: https://github.com/apache/beam/pull/14035#discussion_r591518873
##########
File path: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlOperators.java
##########
@@ -266,7 +275,7 @@ public SqlSyntax getSyntax() {
};
}
- private static RelDataType createSqlType(SqlTypeName typeName, boolean withNullability) {
+ public static RelDataType createSqlType(SqlTypeName typeName, boolean withNullability) {
Review comment:
It was mistakenly done, reverted it to the private.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ibzib commented on pull request #14035: [BEAM-11849] Support ARRAY_CONCAT_AGG fn for Zetasql dialect
Posted by GitBox <gi...@apache.org>.
ibzib commented on pull request #14035:
URL: https://github.com/apache/beam/pull/14035#issuecomment-802361762
Let me make sure this passes internal compliance tests before merging this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] sonam-vend commented on a change in pull request #14035: [BEAM-11849] Support ARRAY_CONCAT_AGG fn for Zetasql dialect
Posted by GitBox <gi...@apache.org>.
sonam-vend commented on a change in pull request #14035:
URL: https://github.com/apache/beam/pull/14035#discussion_r585470992
##########
File path: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlOperators.java
##########
@@ -91,6 +92,12 @@
x -> createTypeFactory().createArrayType(x.getOperandType(0), -1),
new UdafImpl<>(new ArrayAgg.ArrayAggArray()));
+ public static final SqlOperator ARRAY_CONCAT_AGG_FN =
+ createUdafOperator(
+ "array_concat_agg",
+ x -> createTypeFactory().createArrayType(x.getOperandType(0), -1),
Review comment:
@ibzib changing` ARRAY<ARRAY<BIGINT>>` to` ARRAY<BIGINT>` from works, thanks for your help
```
x -> createTypeFactory()
.createArrayType(createTypeFactory().createSqlType(SqlTypeName.BIGINT), -1),
```
##########
File path: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlOperators.java
##########
@@ -91,6 +92,12 @@
x -> createTypeFactory().createArrayType(x.getOperandType(0), -1),
new UdafImpl<>(new ArrayAgg.ArrayAggArray()));
+ public static final SqlOperator ARRAY_CONCAT_AGG_FN =
+ createUdafOperator(
+ "array_concat_agg",
+ x -> createTypeFactory().createArrayType(x.getOperandType(0), -1),
Review comment:
@ibzib changing from ` ARRAY<ARRAY<BIGINT>>` to` ARRAY<BIGINT>` works, thanks for your help
```
x -> createTypeFactory()
.createArrayType(createTypeFactory().createSqlType(SqlTypeName.BIGINT), -1),
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ibzib commented on pull request #14035: [BEAM-11849] Support ARRAY_CONCAT_AGG fn for Zetasql dialect
Posted by GitBox <gi...@apache.org>.
ibzib commented on pull request #14035:
URL: https://github.com/apache/beam/pull/14035#issuecomment-803150149
Unfortunately, this PR fails compliance tests because of nullness issues. I'm guessing it's nothing wrong with this PR, it's probably an unrelated blocker (BEAM-10379).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ibzib commented on a change in pull request #14035: [BEAM-11849] Support ARRAY_CONCAT_AGG fn for Zetasql dialect
Posted by GitBox <gi...@apache.org>.
ibzib commented on a change in pull request #14035:
URL: https://github.com/apache/beam/pull/14035#discussion_r583044780
##########
File path: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlOperators.java
##########
@@ -91,6 +92,12 @@
x -> createTypeFactory().createArrayType(x.getOperandType(0), -1),
new UdafImpl<>(new ArrayAgg.ArrayAggArray()));
+ public static final SqlOperator ARRAY_CONCAT_AGG_FN =
+ createUdafOperator(
+ "array_concat_agg",
+ x -> createTypeFactory().createArrayType(x.getOperandType(0), -1),
Review comment:
I'm pretty sure the problem is with this line here. The return type is supposed to be `ARRAY<BIGINT>`, but the type returned here is `ARRAY<ARRAY<BIGINT>>`. I'm guessing it's because `x.getOperandType(0)` is an array type. Instead it should be the type of elements inside `x.getOperandType(0)`. (I'm not sure exactly what the method is to get that, but you should be able to figure it out using the debugger.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #14035: [BEAM-11849] Support ARRAY_CONCAT_AGG fn for Zetasql dialect
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14035:
URL: https://github.com/apache/beam/pull/14035#issuecomment-1030616570
This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org