You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/11/29 20:52:26 UTC
[GitHub] [beam] Abacn opened a new issue, #24415: [Bug]: Cannot find a matching Calcite SqlTypeName for Beam type: LOGICAL_TYPE seen in 2.44.0 SNAPSHOT
Abacn opened a new issue, #24415:
URL: https://github.com/apache/beam/issues/24415
### What happened?
There is a report of seeing following exception in current Beam 2.44.0 SNAPSHOT. It workes in 2.43.0. Possibly #23891 and/or #24013 break Beam SQL.
```
WARNING: No NameResolverProviders found via ServiceLoader, including for DNS. This is probably due to a broken build. If using ProGuard, check your configuration
Exception in thread "main" java.lang.IllegalArgumentException: Cannot find a matching Calcite SqlTypeName for Beam type: LOGICAL_TYPE
at org.apache.beam.sdk.extensions.sql.impl.utils.CalciteUtils.toSqlTypeName(CalciteUtils.java:198)
at org.apache.beam.sdk.extensions.sql.impl.utils.CalciteUtils.toRelDataType(CalciteUtils.java:305)
at org.apache.beam.sdk.extensions.sql.impl.utils.CalciteUtils.toRelDataType(CalciteUtils.java:312)
at org.apache.beam.sdk.extensions.sql.impl.utils.CalciteUtils.lambda$toCalciteRowType$0(CalciteUtils.java:280)
at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:557)
```
### Issue Priority
Priority: 1
### Issue Component
Component: dsl-sql
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn closed issue #24415: [Bug]: Cannot find a matching Calcite SqlTypeName for Beam type: LOGICAL_TYPE seen in 2.44.0 SNAPSHOT
Posted by GitBox <gi...@apache.org>.
Abacn closed issue #24415: [Bug]: Cannot find a matching Calcite SqlTypeName for Beam type: LOGICAL_TYPE seen in 2.44.0 SNAPSHOT
URL: https://github.com/apache/beam/issues/24415
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #24415: [Bug]: Cannot find a matching Calcite SqlTypeName for Beam type: LOGICAL_TYPE seen in 2.44.0 SNAPSHOT
Posted by GitBox <gi...@apache.org>.
Abacn commented on issue #24415:
URL: https://github.com/apache/beam/issues/24415#issuecomment-1331290614
A breaking change I could imagine is that we removed SqlCharType in #24013. I thought we removed it thoroughly (and replaced with FixedString) but if it is not the case in whatever reason then CalciteUtil will throw exception there.
It is also possible that when the SDK harness container does not bump version (from https://github.com/apache/beam/blob/166b881cb4c2f12983382fcd93c1a33191a0fc23/runners/google-cloud-dataflow-java/build.gradle#L57, the legacy SDK container version is still in August on master branch), When the SNAPSHOT version does not have synchronized SDK harness container, I observed inconsistent state on schema parse and it caused internal PROD integration test failures. Release candidates and released versions do not have this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn closed issue #24415: [Bug]: Cannot find a matching Calcite SqlTypeName for Beam type: LOGICAL_TYPE seen in 2.44.0 SNAPSHOT
Posted by GitBox <gi...@apache.org>.
Abacn closed issue #24415: [Bug]: Cannot find a matching Calcite SqlTypeName for Beam type: LOGICAL_TYPE seen in 2.44.0 SNAPSHOT
URL: https://github.com/apache/beam/issues/24415
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kennknowles commented on issue #24415: [Bug]: Cannot find a matching Calcite SqlTypeName for Beam type: LOGICAL_TYPE seen in 2.44.0 SNAPSHOT
Posted by GitBox <gi...@apache.org>.
kennknowles commented on issue #24415:
URL: https://github.com/apache/beam/issues/24415#issuecomment-1343492601
Merged to 2.44. branch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #24415: [Bug]: Cannot find a matching Calcite SqlTypeName for Beam type: LOGICAL_TYPE seen in 2.44.0 SNAPSHOT
Posted by GitBox <gi...@apache.org>.
Abacn commented on issue #24415:
URL: https://github.com/apache/beam/issues/24415#issuecomment-1334778652
Update: after #24414 it is found the issue is
```
No NameResolverProviders found via ServiceLoader, including for DNS. This is probably due to a broken build.
If using ProGuard, check your configuration Exception in thread "main" java.lang.IllegalArgumentException:
Cannot find a matching Calcite SqlTypeName for Beam logical type: beam:logical_type:micros_instant:v1 at
org.apache.beam.sdk.extensions.sql.impl.utils.CalciteUtils.toSqlTypeName(CalciteUtils.java:206) at
org.apache.beam.sdk.extensions.sql.impl.utils.CalciteUtils.toRelDataType(CalciteUtils.java:319) at
org.apache.beam.sdk.extensions.sql.impl.utils.CalciteUtils.toRelDataType(CalciteUtils.java:326) at
org.apache.beam.sdk.extensions.sql.impl.utils.CalciteUtils.lambda$toCalciteRowType$0(CalciteUtils.java:294)
```
culprit is likely #23969. Previously AvroUtils.toFieldType will reduce to its base type INT64, as it is an unsupported logical type backed by LONG. Now AvroUtils support TimestampMicros explicitly and returns beam logical type SqlTypes.TIMESTAMP (micros_instant), but throughout Beam SQL codebase SqlTypes.TIMESTAMP is not supported.
Need a partial rollback of #23969
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kennknowles closed issue #24415: [Bug]: Cannot find a matching Calcite SqlTypeName for Beam type: LOGICAL_TYPE seen in 2.44.0 SNAPSHOT
Posted by GitBox <gi...@apache.org>.
kennknowles closed issue #24415: [Bug]: Cannot find a matching Calcite SqlTypeName for Beam type: LOGICAL_TYPE seen in 2.44.0 SNAPSHOT
URL: https://github.com/apache/beam/issues/24415
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #24415: [Bug]: Cannot find a matching Calcite SqlTypeName for Beam type: LOGICAL_TYPE seen in 2.44.0 SNAPSHOT
Posted by GitBox <gi...@apache.org>.
Abacn commented on issue #24415:
URL: https://github.com/apache/beam/issues/24415#issuecomment-1337907513
Reopen for the PR of cherry-pick into release-2.44.0 branch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #24415: [Bug]: Cannot find a matching Calcite SqlTypeName for Beam type: LOGICAL_TYPE seen in 2.44.0 SNAPSHOT
Posted by GitBox <gi...@apache.org>.
Abacn commented on issue #24415:
URL: https://github.com/apache/beam/issues/24415#issuecomment-1332808768
Datetime portable logical type was introduced in 2.42.0; decimal introduced in 2.43.0. Reported that the job works as expected for Beam 2.43.0 so that it is not caused by these changes. Between 2.43.0 and 2.44.0 the change is portable BINARY/VARBINARY/CHAR/VARCHAR types. The change added support to these types in `org.apache.beam.sdk.extensions.sql.impl.utils.CalciteUtils.toSqlTypeName`. The only logical type not supported I could imagine is the removed CharType. #24414 should fix it. Close the issue for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org