You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "carl-mastrangelo (via GitHub)" <gi...@apache.org> on 2023/07/26 21:29:44 UTC
[GitHub] [beam] carl-mastrangelo opened a new issue, #27700: [Bug]: Okio present in beam-sdks-java-extensions-sql-jdbc
carl-mastrangelo opened a new issue, #27700:
URL: https://github.com/apache/beam/issues/27700
### What happened?
I am trying to make some HTTP calls from my Java Beam (2.48.0) job. I am using OkHTTP for my HTTP dependency (4.11.0) which has a dependency on Okio (3.2.0). The Jar `beam-sdks-java-extensions-sql-jdbc` includes a pretty old version of Okio (1.6.0), which happens to conflict with the version needed by OkHTTP.
As a result, the later stages of my Beam job fail with class loader errors since the beam jar is earlier on the classpath (error included below).
```
org.apache.beam.sdk.util.UserCodeException: java.lang.NoClassDefFoundError: Could not initialize class okhttp3.internal.concurrent.TaskRunner
at org.apache.beam.sdk.util.UserCodeException.wrap ( org/apache.beam.sdk.util/UserCodeException.java:39 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:799 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:313 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:245 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:1788 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:142 )
at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:2506 )
at org.apache.beam.sdk.transforms.Filter$1.processElement ( org/apache.beam.sdk.transforms/Filter.java:211 )
at org.apache.beam.sdk.transforms.Filter$1$DoFnInvoker.invokeProcessElement
at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:799 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:313 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:245 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:1788 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:142 )
at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:2506 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:799 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:313 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:245 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:1788 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:142 )
at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:2506 )
at org.apache.beam.sdk.io.FileIO$ReadMatches$ToReadableFileFn.process ( org/apache.beam.sdk.io/FileIO.java:872 )
at org.apache.beam.sdk.io.FileIO$ReadMatches$ToReadableFileFn$DoFnInvoker.invokeProcessElement
at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:799 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:313 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:245 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:1788 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:142 )
at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:2506 )
at org.apache.beam.sdk.transforms.MapElements$2.processElement ( org/apache.beam.sdk.transforms/MapElements.java:151 )
at org.apache.beam.sdk.transforms.MapElements$2$DoFnInvoker.invokeProcessElement
at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:799 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:313 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:245 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:1788 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:142 )
at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.outputWithTimestamp ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:2523 )
at org.apache.beam.sdk.transforms.Reify$ExtractTimestampsFromValues$1.processElement ( org/apache.beam.sdk.transforms/Reify.java:206 )
at org.apache.beam.sdk.transforms.Reify$ExtractTimestampsFromValues$1$DoFnInvoker.invokeProcessElement
at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:799 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:313 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:245 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:1788 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:142 )
at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:2506 )
at org.apache.beam.sdk.transforms.ReifyTimestamps$RemoveWildcard$1.process ( org/apache.beam.sdk.transforms/ReifyTimestamps.java:70 )
at org.apache.beam.sdk.transforms.ReifyTimestamps$RemoveWildcard$1$DoFnInvoker.invokeProcessElement
at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:799 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:313 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:245 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:1788 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:142 )
at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:2506 )
at org.apache.beam.sdk.transforms.Reshuffle$1.processElement ( org/apache.beam.sdk.transforms/Reshuffle.java:101 )
at org.apache.beam.sdk.transforms.Reshuffle$1$DoFnInvoker.invokeProcessElement
at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( org/apache.beam.fn.harness/FnApiDoFnRunner.java:799 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:313 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( org/apache.beam.fn.harness.data/PCollectionConsumerRegistry.java:245 )
at org.apache.beam.fn.harness.BeamFnDataReadRunner.forwardElementToConsumer ( org/apache.beam.fn.harness/BeamFnDataReadRunner.java:213 )
at org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.multiplexElements ( org/apache.beam.sdk.fn.data/BeamFnDataInboundObserver.java:158 )
at org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.awaitCompletion ( org/apache.beam.sdk.fn.data/BeamFnDataInboundObserver.java:123 )
at org.apache.beam.fn.harness.control.ProcessBundleHandler.processBundle ( org/apache.beam.fn.harness.control/ProcessBundleHandler.java:550 )
at org.apache.beam.fn.harness.control.BeamFnControlClient.delegateOnInstructionRequestType ( org/apache.beam.fn.harness.control/BeamFnControlClient.java:151 )
at org.apache.beam.fn.harness.control.BeamFnControlClient$InboundObserver.lambda$onNext$0 ( org/apache.beam.fn.harness.control/BeamFnControlClient.java:116 )
at java.util.concurrent.Executors$RunnableAdapter.call ( java/util.concurrent/Executors.java:539 )
at java.util.concurrent.FutureTask.run ( java/util.concurrent/FutureTask.java:264 )
at org.apache.beam.sdk.util.UnboundedScheduledExecutorService$ScheduledFutureTask.run ( org/apache.beam.sdk.util/UnboundedScheduledExecutorService.java:163 )
at java.util.concurrent.ThreadPoolExecutor.runWorker ( java/util.concurrent/ThreadPoolExecutor.java:1136 )
at java.util.concurrent.ThreadPoolExecutor$Worker.run ( java/util.concurrent/ThreadPoolExecutor.java:635 )
at java.lang.Thread.run ( java/lang/Thread.java:833 )
Caused by: java.lang.NoClassDefFoundError
at okhttp3.ConnectionPool.<init> ( ConnectionPool.kt:41 )
at okhttp3.ConnectionPool.<init> ( ConnectionPool.kt:47 )
at okhttp3.OkHttpClient$Builder.<init> ( OkHttpClient.kt:471 )
Caused by: java.lang.ExceptionInInitializerError
at okhttp3.internal.Util.<clinit> ( Util.kt:71 )
at okhttp3.internal.concurrent.TaskRunner.<clinit> ( TaskRunner.kt:309 )
at okhttp3.ConnectionPool.<init> ( ConnectionPool.kt:41 )
at okhttp3.ConnectionPool.<init> ( ConnectionPool.kt:47 )
at okhttp3.OkHttpClient$Builder.<init> ( OkHttpClient.kt:471 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( FnApiDoFnRunner.java:799 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:313 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:245 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( FnApiDoFnRunner.java:1788 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( FnApiDoFnRunner.java:142 )
at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( FnApiDoFnRunner.java:2506 )
at org.apache.beam.sdk.transforms.Filter$1.processElement ( Filter.java:211 )
at org.apache.beam.sdk.transforms.Filter$1$DoFnInvoker.invokeProcessElement
at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( FnApiDoFnRunner.java:799 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:313 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:245 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( FnApiDoFnRunner.java:1788 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( FnApiDoFnRunner.java:142 )
at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( FnApiDoFnRunner.java:2506 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( FnApiDoFnRunner.java:799 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:313 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:245 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( FnApiDoFnRunner.java:1788 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( FnApiDoFnRunner.java:142 )
at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( FnApiDoFnRunner.java:2506 )
at org.apache.beam.sdk.io.FileIO$ReadMatches$ToReadableFileFn.process ( FileIO.java:872 )
at org.apache.beam.sdk.io.FileIO$ReadMatches$ToReadableFileFn$DoFnInvoker.invokeProcessElement
at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( FnApiDoFnRunner.java:799 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:313 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:245 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( FnApiDoFnRunner.java:1788 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( FnApiDoFnRunner.java:142 )
at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( FnApiDoFnRunner.java:2506 )
at org.apache.beam.sdk.transforms.MapElements$2.processElement ( MapElements.java:151 )
at org.apache.beam.sdk.transforms.MapElements$2$DoFnInvoker.invokeProcessElement
at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( FnApiDoFnRunner.java:799 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:313 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:245 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( FnApiDoFnRunner.java:1788 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( FnApiDoFnRunner.java:142 )
at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.outputWithTimestamp ( FnApiDoFnRunner.java:2523 )
at org.apache.beam.sdk.transforms.Reify$ExtractTimestampsFromValues$1.processElement ( Reify.java:206 )
at org.apache.beam.sdk.transforms.Reify$ExtractTimestampsFromValues$1$DoFnInvoker.invokeProcessElement
at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( FnApiDoFnRunner.java:799 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:313 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:245 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( FnApiDoFnRunner.java:1788 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( FnApiDoFnRunner.java:142 )
at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( FnApiDoFnRunner.java:2506 )
at org.apache.beam.sdk.transforms.ReifyTimestamps$RemoveWildcard$1.process ( ReifyTimestamps.java:70 )
at org.apache.beam.sdk.transforms.ReifyTimestamps$RemoveWildcard$1$DoFnInvoker.invokeProcessElement
at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( FnApiDoFnRunner.java:799 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:313 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:245 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo ( FnApiDoFnRunner.java:1788 )
at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000 ( FnApiDoFnRunner.java:142 )
at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output ( FnApiDoFnRunner.java:2506 )
at org.apache.beam.sdk.transforms.Reshuffle$1.processElement ( Reshuffle.java:101 )
at org.apache.beam.sdk.transforms.Reshuffle$1$DoFnInvoker.invokeProcessElement
at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo ( FnApiDoFnRunner.java:799 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:313 )
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept ( PCollectionConsumerRegistry.java:245 )
at org.apache.beam.fn.harness.BeamFnDataReadRunner.forwardElementToConsumer ( BeamFnDataReadRunner.java:213 )
at org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.multiplexElements ( BeamFnDataInboundObserver.java:158 )
at org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.awaitCompletion ( BeamFnDataInboundObserver.java:123 )
at org.apache.beam.fn.harness.control.ProcessBundleHandler.processBundle ( ProcessBundleHandler.java:550 )
at org.apache.beam.fn.harness.control.BeamFnControlClient.delegateOnInstructionRequestType ( BeamFnControlClient.java:151 )
at org.apache.beam.fn.harness.control.BeamFnControlClient$InboundObserver.lambda$onNext$0 ( BeamFnControlClient.java:116 )
at java.util.concurrent.Executors$RunnableAdapter.call ( Executors.java:539 )
at java.util.concurrent.FutureTask.run ( FutureTask.java:264 )
at org.apache.beam.sdk.util.UnboundedScheduledExecutorService$ScheduledFutureTask.run ( UnboundedScheduledExecutorService.java:163 )
at java.util.concurrent.ThreadPoolExecutor.runWorker ( ThreadPoolExecutor.java:1136 )
at java.util.concurrent.ThreadPoolExecutor$Worker.run ( ThreadPoolExecutor.java:635 )
at java.lang.Thread.run ( Thread.java:833 )
```
### Issue Priority
Priority: 3 (minor)
### Issue Components
- [ ] Component: Python SDK
- [X] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] carl-mastrangelo commented on issue #27700: [Bug]: Okio present in beam-sdks-java-extensions-sql-jdbc
Posted by "carl-mastrangelo (via GitHub)" <gi...@apache.org>.
carl-mastrangelo commented on issue #27700:
URL: https://github.com/apache/beam/issues/27700#issuecomment-1654589941
> What functionality are you use for beam-sdks-java-extensions-sql-jdbc ?
It may be the incorrect jar to use, but I need to write the result of my HTTP calls to a database. In effect, GET a resource and then save it to a DB.
> I see, its because beam-sdks-java-extensions-sql-jdbc is a fat" jar (having all dependency shaded)
It is a fat jar, but I don't believe it is shaded. I think that would help though. (If it were shaded, the packages would look more like `org.apache.beam.vendored.okio. BufferedSink`). However, any one who was accidentally depending on this jar would also fail at runtime with a NoClassDefError if the class was shaded. Personally, I think shading would be the reasonable choice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Okio present in beam-sdks-java-extensions-sql-jdbc [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #27700:
URL: https://github.com/apache/beam/issues/27700#issuecomment-5890703060
This issue has been closed due to lack of activity. If you think that is incorrect, you can reopen the issue at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #27700: [Bug]: Okio present in beam-sdks-java-extensions-sql-jdbc
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #27700:
URL: https://github.com/apache/beam/issues/27700#issuecomment-1654790266
Are you using Beam's JdbcIO or Beam SQL to write data to database? If just using JdbcIO, then beam-sdks-jaca-io-jdbc is the corrext dependency to use. The extension-sql-jdbc is used to run a self contained service for beam SQL (it has a main function even) so usually not meant to be used as dependency
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #27700: [Bug]: Okio present in beam-sdks-java-extensions-sql-jdbc
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #27700:
URL: https://github.com/apache/beam/issues/27700#issuecomment-1654268682
Dependency tree:
```
+--- project :sdks:java:extensions:sql
| +--- project :sdks:java:io:parquet
| | +--- project :sdks:java:io:hadoop-common
| | | +--- org.apache.hadoop:hadoop-client:2.10.2
| | | | +--- org.apache.hadoop:hadoop-hdfs-client:2.10.2
| | | | | \--- com.squareup.okhttp:okhttp:2.7.5
```
If you're not using hadoop, probably it is safe to overwrite okhttp to 3 in your project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #27700: [Bug]: Okio present in beam-sdks-java-extensions-sql-jdbc
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #27700:
URL: https://github.com/apache/beam/issues/27700#issuecomment-1654524401
Okio is from OkHttp:
```
| | | | +--- org.apache.hadoop:hadoop-hdfs-client:2.10.2
| | | | | \--- com.squareup.okhttp:okhttp:2.7.5
| | | | | \--- com.squareup.okio:okio:1.6.0
```
I see, its because beam-sdks-java-extensions-sql-jdbc is a fat" jar (having all dependency shaded) so one cannot overwrite its dependencies. What functionality are you use for beam-sdks-java-extensions-sql-jdbc ? If it's not used then don't include it in classpath; if it is used probably need a custom build. This module (sql-extensions-jdbc) is not actively maintained
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] carl-mastrangelo commented on issue #27700: [Bug]: Okio present in beam-sdks-java-extensions-sql-jdbc
Posted by "carl-mastrangelo (via GitHub)" <gi...@apache.org>.
carl-mastrangelo commented on issue #27700:
URL: https://github.com/apache/beam/issues/27700#issuecomment-1654441784
Unfortunately, it still pollutes the class path:
```
$ unzip -l beam-sdks-java-extensions-sql-jdbc-2.48.0.jar | grep \ ok
0 02-01-1980 00:00 okio/
2046 02-01-1980 00:00 okio/AsyncTimeout$1.class
1823 02-01-1980 00:00 okio/AsyncTimeout$2.class
750 02-01-1980 00:00 okio/AsyncTimeout$Watchdog.class
3546 02-01-1980 00:00 okio/AsyncTimeout.class
3385 02-01-1980 00:00 okio/Base64.class
1213 02-01-1980 00:00 okio/Buffer$1.class
1310 02-01-1980 00:00 okio/Buffer$2.class
28779 02-01-1980 00:00 okio/Buffer.class
1314 02-01-1980 00:00 okio/BufferedSink.class
1598 02-01-1980 00:00 okio/BufferedSource.class
9275 02-01-1980 00:00 okio/ByteString.class
3600 02-01-1980 00:00 okio/DeflaterSink.class
1482 02-01-1980 00:00 okio/ForwardingSink.class
1391 02-01-1980 00:00 okio/ForwardingSource.class
1680 02-01-1980 00:00 okio/ForwardingTimeout.class
3213 02-01-1980 00:00 okio/GzipSink.class
4637 02-01-1980 00:00 okio/GzipSource.class
3359 02-01-1980 00:00 okio/InflaterSource.class
2011 02-01-1980 00:00 okio/Okio$1.class
1950 02-01-1980 00:00 okio/Okio$2.class
2017 02-01-1980 00:00 okio/Okio$3.class
4352 02-01-1980 00:00 okio/Okio.class
1738 02-01-1980 00:00 okio/RealBufferedSink$1.class
6950 02-01-1980 00:00 okio/RealBufferedSink.class
2054 02-01-1980 00:00 okio/RealBufferedSource$1.class
9757 02-01-1980 00:00 okio/RealBufferedSource.class
2246 02-01-1980 00:00 okio/Segment.class
5960 02-01-1980 00:00 okio/SegmentedByteString.class
1105 02-01-1980 00:00 okio/SegmentPool.class
323 02-01-1980 00:00 okio/Sink.class
275 02-01-1980 00:00 okio/Source.class
728 02-01-1980 00:00 okio/Timeout$1.class
2296 02-01-1980 00:00 okio/Timeout.class
2098 02-01-1980 00:00 okio/Util.class
```
Also to be clear, It's Okio, not OkHttp directly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Okio present in beam-sdks-java-extensions-sql-jdbc [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #27700:
URL: https://github.com/apache/beam/issues/27700#issuecomment-5468739660
This issue has been marked as stale due to 150 days of inactivity. It will be closed in 30 days if no further activity occurs. If you think that’s incorrect or this issue still needs to be addressed, please simply write any comment. If closed, you can reopen the issue at any time. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Okio present in beam-sdks-java-extensions-sql-jdbc [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #27700: [Bug]: Okio present in beam-sdks-java-extensions-sql-jdbc
URL: https://github.com/apache/beam/issues/27700
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org