You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/04 18:11:04 UTC
[GitHub] [beam] damccorm opened a new issue, #20562: Bigquery readTableRowsWithSchema fails with IllegalStateException
damccorm opened a new issue, #20562:
URL: https://github.com/apache/beam/issues/20562
If you are templating the pipeline and use the function BigQueryIO.readTableRowsWithSchema(), it fails with the following exception even though no .get() was called on the ValueProvider
```
An exception occured while executing the Java class. Value only available at runtime, but accessed
from a non-runtime context: RuntimeValueProvider
```
The following stack trace gives more clue and the problem seems to be that BigQueryQuerySourceDef is trying to call a .get on ValueProvider which is wrong. Here is the exact code line in master
[https://github.com/apache/beam/blame/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryQuerySourceDef.java#L103](https://github.com/apache/beam/blame/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryQuerySourceDef.java#L103)
```
java.lang.IllegalStateException: Value only available at runtime, but accessed from a non-runtime context:
RuntimeValueProvider{propertyName=tablename, default=null}
at org.apache.beam.sdk.options.ValueProvider$RuntimeValueProvider.get
(ValueProvider.java:250)
at org.apache.beam.sdk.options.ValueProvider$NestedValueProvider.get (ValueProvider.java:141)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryQuerySourceDef.getBeamSchema (BigQueryQuerySourceDef.java:163)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO$TypedRead.expand (BigQueryIO.java:1188)
at
org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO$TypedRead.expand (BigQueryIO.java:701)
at org.apache.beam.sdk.Pipeline.applyInternal
(Pipeline.java:542)
at org.apache.beam.sdk.Pipeline.applyTransform (Pipeline.java:493)
at
org.apache.beam.sdk.values.PBegin.apply (PBegin.java:56)
at org.apache.beam.sdk.Pipeline.apply
(Pipeline.java:186)
at com.abc.pipeline.main (BomboraIntentCrmIdPipeline.java:93)
at sun.reflect.NativeMethodAccessorImpl.invoke0
(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at
java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run
(ExecJavaMojo.java:282)
at java.lang.Thread.run (Thread.java:836)
```
Imported from Jira [BEAM-11060](https://issues.apache.org/jira/browse/BEAM-11060). Original Jira may contain additional context.
Reported by: bsikander.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kennknowles closed issue #20562: Bigquery readTableRowsWithSchema fails with IllegalStateException
Posted by GitBox <gi...@apache.org>.
kennknowles closed issue #20562: Bigquery readTableRowsWithSchema fails with IllegalStateException
URL: https://github.com/apache/beam/issues/20562
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kennknowles commented on issue #20562: Bigquery readTableRowsWithSchema fails with IllegalStateException
Posted by GitBox <gi...@apache.org>.
kennknowles commented on issue #20562:
URL: https://github.com/apache/beam/issues/20562#issuecomment-1334587147
and the simple fact is that the valueprovider is dereferenced eagerly and that's that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kennknowles commented on issue #20562: Bigquery readTableRowsWithSchema fails with IllegalStateException
Posted by GitBox <gi...@apache.org>.
kennknowles commented on issue #20562:
URL: https://github.com/apache/beam/issues/20562#issuecomment-1334587375
ValueProviders are only dataflow and only classic templates so there is not going to be any changes to using them. If anything we will remove them generally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kennknowles commented on issue #20562: Bigquery readTableRowsWithSchema fails with IllegalStateException
Posted by GitBox <gi...@apache.org>.
kennknowles commented on issue #20562:
URL: https://github.com/apache/beam/issues/20562#issuecomment-1334586897
Since the link was not a permalink it is wrong now. It is https://github.com/apache/beam/blame/b9f2055897b6dffd472125ece50609ecf70f5db4/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryQuerySourceDef.java#L166
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org