You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Beam JIRA Bot (Jira)" <ji...@apache.org> on 2022/03/16 17:26:00 UTC
[jira] [Commented] (BEAM-12881) Issue with Beam JDBC IO for GCP Dataflow
[ https://issues.apache.org/jira/browse/BEAM-12881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17507757#comment-17507757 ]
Beam JIRA Bot commented on BEAM-12881:
--------------------------------------
This issue is P2 but has been unassigned without any comment for 60 days so it has been labeled "stale-P2". If this issue is still affecting you, we care! Please comment and remove the label. Otherwise, in 14 days the issue will be moved to P3.
Please see https://beam.apache.org/contribute/jira-priorities/ for a detailed explanation of what these priorities mean.
> Issue with Beam JDBC IO for GCP Dataflow
> ----------------------------------------
>
> Key: BEAM-12881
> URL: https://issues.apache.org/jira/browse/BEAM-12881
> Project: Beam
> Issue Type: Bug
> Components: cross-language, io-java-jdbc
> Affects Versions: 2.31.0
> Environment: python 3.7.3
> openjdk version "11.0.12" 2021-07-20
> OpenJDK Runtime Environment (build 11.0.12+7-post-Debian-2deb10u1)
> Reporter: Gaurav Pathak
> Priority: P2
> Labels: GCP, beam, dataflow, jdbc_connector, python3, stale-P2
>
> Hi,
> We have a dataflow job in GCP that we are hoping to use PostgreSQL as our sink for streaming data. Currently we are trying to connect to instance using apache_beam.io.jdbc(default expansion service) but we are constantly getting error while trying to create the tempate for dataflow job. The error seems to come at *"INFO: Expanding 'load to sql' with URN 'beam:external:java:schemaio:jdbc:write:v1'"*
>
>
> Below is the code and error log.
> {code:java}
> Connection portion.
> load_data = data | "load to sql" >> WriteToJdbc(
> table_name='test',
> driver_class_name='org.postgresql.Driver',
> jdbc_url='jdbc:{}://{}:{}/{}'.format("postgresql", "ip", "5432", "postgres"),
> username='test_user',
> password='password'
> )
> Error:
> INFO:root:Dataflow job is running on dev INFO:root:Job Run Id is 3618 INFO:root:Job Name is validation-parser-20210913-17-46 INFO:apache_beam.utils.subprocess_server:Using cached job server jar from https://repo.maven.apache.org/maven2/org/apache/beam/beam-sdks-java-extensions-schemaio-expansion-service/2.31.0/beam-sdks-java-extensions-schemaio-expansion-service-2.31.0.jar INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '/home/user/.apache_beam/cache/jars/beam-sdks-java-extensions-schemaio-expansion-service-2.31.0.jar' '39547'] INFO:apache_beam.utils.subprocess_server:b'Starting expansion service at localhost:39547' INFO:apache_beam.utils.subprocess_server:b'Sep 13, 2021 5:46:34 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms' INFO:apache_beam.utils.subprocess_server:b'INFO: Registering external transforms: [beam:external:java:schemaio:jdbc:read:v1, beam:external:java:schemaio:jdbc:write:v1, beam:external:java:schemaio:avro:read:v1, beam:external:java:schemaio:avro:write:v1, beam:external:java:generate_sequence:v1]' INFO:apache_beam.utils.subprocess_server:b'\tbeam:external:java:schemaio:jdbc:read:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$11/0x000000084007f840@13b6d03' INFO:apache_beam.utils.subprocess_server:b'\tbeam:external:java:schemaio:jdbc:write:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$11/0x000000084007f840@f5f2bb7' INFO:apache_beam.utils.subprocess_server:b'\tbeam:external:java:schemaio:avro:read:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$11/0x000000084007f840@73035e27' INFO:apache_beam.utils.subprocess_server:b'\tbeam:external:java:schemaio:avro:write:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$11/0x000000084007f840@64c64813' INFO:apache_beam.utils.subprocess_server:b'\tbeam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$11/0x000000084007f840@3ecf72fd' INFO:apache_beam.utils.subprocess_server:b'Sep 13, 2021 5:46:35 PM org.apache.beam.sdk.expansion.service.ExpansionService expand' INFO:apache_beam.utils.subprocess_server:b"INFO: Expanding 'load to sql' with URN 'beam:external:java:schemaio:jdbc:write:v1'" ERROR:root:Failed to launch datapipeline. Error org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: Unexpected type_info: TYPEINFO_NOT_SET at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2050) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958) at org.apache.beam.runners.core.construction.RehydratedComponents.getPCollection(RehydratedComponents.java:139) at org.apache.beam.sdk.expansion.service.ExpansionService.lambda$expand$0(ExpansionService.java:425) at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:178) at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) at java.base/java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet.lambda$entryConsumer$0(Collections.java:1576) at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133) at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) at java.base/java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet$UnmodifiableEntrySetSpliterator.forEachRemaining(Collections.java:1601) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) at org.apache.beam.sdk.expansion.service.ExpansionService.expand(ExpansionService.java:420) at org.apache.beam.sdk.expansion.service.ExpansionService.expand(ExpansionService.java:494) at org.apache.beam.model.expansion.v1.ExpansionServiceGrpc$MethodHandlers.invoke(ExpansionServiceGrpc.java:219) at org.apache.beam.vendor.grpc.v1p36p0.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182) at org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:331) at org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:797) at org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) at org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: Unexpected type_info: TYPEINFO_NOT_SET at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2050) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958) at org.apache.beam.runners.core.construction.RehydratedComponents.getCoder(RehydratedComponents.java:168) at org.apache.beam.runners.core.construction.PCollectionTranslation.fromProto(PCollectionTranslation.java:51) at org.apache.beam.runners.core.construction.RehydratedComponents$3.load(RehydratedComponents.java:108) at org.apache.beam.runners.core.construction.RehydratedComponents$3.load(RehydratedComponents.java:98) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044) ... 27 more Caused by: java.lang.IllegalArgumentException: Unexpected type_info: TYPEINFO_NOT_SET at org.apache.beam.sdk.schemas.SchemaTranslation.fieldTypeFromProtoWithoutNullable(SchemaTranslation.java:362) at org.apache.beam.sdk.schemas.SchemaTranslation.fieldTypeFromProto(SchemaTranslation.java:261) at org.apache.beam.sdk.schemas.SchemaTranslation.fieldTypeFromProtoWithoutNullable(SchemaTranslation.java:358) at org.apache.beam.sdk.schemas.SchemaTranslation.fieldTypeFromProto(SchemaTranslation.java:261) at org.apache.beam.sdk.schemas.SchemaTranslation.fieldFromProto(SchemaTranslation.java:255) at org.apache.beam.sdk.schemas.SchemaTranslation.schemaFromProto(SchemaTranslation.java:229) at org.apache.beam.runners.core.construction.CoderTranslators$8.fromComponents(CoderTranslators.java:169) at org.apache.beam.runners.core.construction.CoderTranslators$8.fromComponents(CoderTranslators.java:151) at org.apache.beam.runners.core.construction.CoderTranslation.fromKnownCoder(CoderTranslation.java:170) at org.apache.beam.runners.core.construction.CoderTranslation.fromProto(CoderTranslation.java:145) at org.apache.beam.runners.core.construction.RehydratedComponents$2.load(RehydratedComponents.java:87) at org.apache.beam.runners.core.construction.RehydratedComponents$2.load(RehydratedComponents.java:82) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044) ... 38 more Traceback (most recent call last): File "muthu_sql.py", line 921, in run password='E9d3X5h' File "/home/user/venv/lib/python3.7/site-packages/apache_beam/pvalue.py", line 136, in __or__ return self.pipeline.apply(ptransform, self) File "/home/user/venv/lib/python3.7/site-packages/apache_beam/pipeline.py", line 641, in apply transform.transform, pvalueish, label or transform.label) File "/home/user/venv/lib/python3.7/site-packages/apache_beam/pipeline.py", line 651, in apply return self.apply(transform, pvalueish) File "/home/user/venv/lib/python3.7/site-packages/apache_beam/pipeline.py", line 694, in apply pvalueish_result = self.runner.apply(transform, pvalueish, self._options) File "/home/user/venv/lib/python3.7/site-packages/apache_beam/runners/dataflow/dataflow_runner.py", line 141, in apply return super(DataflowRunner, self).apply(transform, input, options) File "/home/user/venv/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 185, in apply return m(transform, input, options) File "/home/user/venv/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 215, in apply_PTransform return transform.expand(input) File "/home/user/venv/lib/python3.7/site-packages/apache_beam/transforms/external.py", line 297, in expand raise RuntimeError(response.error) RuntimeError: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: Unexpected type_info: TYPEINFO_NOT_SET at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2050) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958) at org.apache.beam.runners.core.construction.RehydratedComponents.getPCollection(RehydratedComponents.java:139) at org.apache.beam.sdk.expansion.service.ExpansionService.lambda$expand$0(ExpansionService.java:425) at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:178) at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) at java.base/java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet.lambda$entryConsumer$0(Collections.java:1576) at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133) at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) at java.base/java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet$UnmodifiableEntrySetSpliterator.forEachRemaining(Collections.java:1601) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) at org.apache.beam.sdk.expansion.service.ExpansionService.expand(ExpansionService.java:420) at org.apache.beam.sdk.expansion.service.ExpansionService.expand(ExpansionService.java:494) at org.apache.beam.model.expansion.v1.ExpansionServiceGrpc$MethodHandlers.invoke(ExpansionServiceGrpc.java:219) at org.apache.beam.vendor.grpc.v1p36p0.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182) at org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:331) at org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:797) at org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) at org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: Unexpected type_info: TYPEINFO_NOT_SET at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2050) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958) at org.apache.beam.runners.core.construction.RehydratedComponents.getCoder(RehydratedComponents.java:168) at org.apache.beam.runners.core.construction.PCollectionTranslation.fromProto(PCollectionTranslation.java:51) at org.apache.beam.runners.core.construction.RehydratedComponents$3.load(RehydratedComponents.java:108) at org.apache.beam.runners.core.construction.RehydratedComponents$3.load(RehydratedComponents.java:98) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044) ... 27 more Caused by: java.lang.IllegalArgumentException: Unexpected type_info: TYPEINFO_NOT_SET at org.apache.beam.sdk.schemas.SchemaTranslation.fieldTypeFromProtoWithoutNullable(SchemaTranslation.java:362) at org.apache.beam.sdk.schemas.SchemaTranslation.fieldTypeFromProto(SchemaTranslation.java:261) at org.apache.beam.sdk.schemas.SchemaTranslation.fieldTypeFromProtoWithoutNullable(SchemaTranslation.java:358) at org.apache.beam.sdk.schemas.SchemaTranslation.fieldTypeFromProto(SchemaTranslation.java:261) at org.apache.beam.sdk.schemas.SchemaTranslation.fieldFromProto(SchemaTranslation.java:255) at org.apache.beam.sdk.schemas.SchemaTranslation.schemaFromProto(SchemaTranslation.java:229) at org.apache.beam.runners.core.construction.CoderTranslators$8.fromComponents(CoderTranslators.java:169) at org.apache.beam.runners.core.construction.CoderTranslators$8.fromComponents(CoderTranslators.java:151) at org.apache.beam.runners.core.construction.CoderTranslation.fromKnownCoder(CoderTranslation.java:170) at org.apache.beam.runners.core.construction.CoderTranslation.fromProto(CoderTranslation.java:145) at org.apache.beam.runners.core.construction.RehydratedComponents$2.load(RehydratedComponents.java:87) at org.apache.beam.runners.core.construction.RehydratedComponents$2.load(RehydratedComponents.java:82) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044) ... 38 moreTraceback (most recent call last): File "muthu_sql.py", line 985, in <module> main(args.env, args.run_date) File "muthu_sql.py", line 970, in main run() File "muthu_sql.py", line 921, in run password='E9d3X5h' File "/home/user/venv/lib/python3.7/site-packages/apache_beam/pvalue.py", line 136, in __or__ return self.pipeline.apply(ptransform, self) File "/home/user/venv/lib/python3.7/site-packages/apache_beam/pipeline.py", line 641, in apply transform.transform, pvalueish, label or transform.label) File "/home/user/venv/lib/python3.7/site-packages/apache_beam/pipeline.py", line 651, in apply return self.apply(transform, pvalueish) File "/home/user/venv/lib/python3.7/site-packages/apache_beam/pipeline.py", line 694, in apply pvalueish_result = self.runner.apply(transform, pvalueish, self._options) File "/home/user/venv/lib/python3.7/site-packages/apache_beam/runners/dataflow/dataflow_runner.py", line 141, in apply return super(DataflowRunner, self).apply(transform, input, options) File "/home/user/venv/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 185, in apply return m(transform, input, options) File "/home/user/venv/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 215, in apply_PTransform return transform.expand(input) File "/home/user/venv/lib/python3.7/site-packages/apache_beam/transforms/external.py", line 297, in expand raise RuntimeError(response.error) RuntimeError: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: Unexpected type_info: TYPEINFO_NOT_SET at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2050) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958) at org.apache.beam.runners.core.construction.RehydratedComponents.getPCollection(RehydratedComponents.java:139) at org.apache.beam.sdk.expansion.service.ExpansionService.lambda$expand$0(ExpansionService.java:425) at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:178) at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) at java.base/java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet.lambda$entryConsumer$0(Collections.java:1576) at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133) at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) at java.base/java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet$UnmodifiableEntrySetSpliterator.forEachRemaining(Collections.java:1601) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) at org.apache.beam.sdk.expansion.service.ExpansionService.expand(ExpansionService.java:420) at org.apache.beam.sdk.expansion.service.ExpansionService.expand(ExpansionService.java:494) at org.apache.beam.model.expansion.v1.ExpansionServiceGrpc$MethodHandlers.invoke(ExpansionServiceGrpc.java:219) at org.apache.beam.vendor.grpc.v1p36p0.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182) at org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:331) at org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:797) at org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) at org.apache.beam.vendor.grpc.v1p36p0.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: Unexpected type_info: TYPEINFO_NOT_SET at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2050) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958) at org.apache.beam.runners.core.construction.RehydratedComponents.getCoder(RehydratedComponents.java:168) at org.apache.beam.runners.core.construction.PCollectionTranslation.fromProto(PCollectionTranslation.java:51) at org.apache.beam.runners.core.construction.RehydratedComponents$3.load(RehydratedComponents.java:108) at org.apache.beam.runners.core.construction.RehydratedComponents$3.load(RehydratedComponents.java:98) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044) ... 27 more Caused by: java.lang.IllegalArgumentException: Unexpected type_info: TYPEINFO_NOT_SET at org.apache.beam.sdk.schemas.SchemaTranslation.fieldTypeFromProtoWithoutNullable(SchemaTranslation.java:362) at org.apache.beam.sdk.schemas.SchemaTranslation.fieldTypeFromProto(SchemaTranslation.java:261) at org.apache.beam.sdk.schemas.SchemaTranslation.fieldTypeFromProtoWithoutNullable(SchemaTranslation.java:358) at org.apache.beam.sdk.schemas.SchemaTranslation.fieldTypeFromProto(SchemaTranslation.java:261) at org.apache.beam.sdk.schemas.SchemaTranslation.fieldFromProto(SchemaTranslation.java:255) at org.apache.beam.sdk.schemas.SchemaTranslation.schemaFromProto(SchemaTranslation.java:229) at org.apache.beam.runners.core.construction.CoderTranslators$8.fromComponents(CoderTranslators.java:169) at org.apache.beam.runners.core.construction.CoderTranslators$8.fromComponents(CoderTranslators.java:151) at org.apache.beam.runners.core.construction.CoderTranslation.fromKnownCoder(CoderTranslation.java:170) at org.apache.beam.runners.core.construction.CoderTranslation.fromProto(CoderTranslation.java:145) at org.apache.beam.runners.core.construction.RehydratedComponents$2.load(RehydratedComponents.java:87) at org.apache.beam.runners.core.construction.RehydratedComponents$2.load(RehydratedComponents.java:82) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044) ... 38 more(venv) user@cloudshell:~/sql_test (edp-dev-eds)$
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)