You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Beam JIRA Bot (Jira)" <ji...@apache.org> on 2021/05/10 17:20:01 UTC
[jira] [Commented] (BEAM-10516) Including TableProvider IO
dependencies as `provided` can lead to ClassDefNotFoundErrors for users
[ https://issues.apache.org/jira/browse/BEAM-10516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342030#comment-17342030 ]
Beam JIRA Bot commented on BEAM-10516:
--------------------------------------
This issue was marked "stale-P2" and has not received a public comment in 14 days. It is now automatically moved to P3. If you are still affected by it, you can comment and move it back to P2.
> Including TableProvider IO dependencies as `provided` can lead to ClassDefNotFoundErrors for users
> --------------------------------------------------------------------------------------------------
>
> Key: BEAM-10516
> URL: https://issues.apache.org/jira/browse/BEAM-10516
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql, sdk-java-core
> Reporter: Scott Lukas
> Priority: P3
> Labels: Clarified, beam-fixit
>
> The SQL extension uses ServiceLoader to load table providers: https://github.com/apache/beam/blob/v2.22.0/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamCalciteSchemaFactory.java#L85
> And the IOs used in the built-in table providers (e.g. pubsub json, avro, parquet, ...) are included as {{provided}} dependencies, not {{compile}}. This means that we are able to build the sql extension jar, and AutoService picks them up and records them in META-INF/services. However because they are {{provided}} dependencies, the necessary IO modules are not marked as dependencies for the SQL extension.
> This means consumers of :sdks:java:extensions:sql (like :sdks:java:extensions:sql:jdbc) can end up with ServiceLoader registering a TableProvider that will fail with ClassDefNotFoundError, unless they add the necessary dependency.
> Possible solution: Refactor TableProvider to beam core (or another shared module) in order to solve Service Loader errors with SchemaCapableIOTableProviderWrapper. Remove getTableType overrides of SchemaCapableIOProviderWrapper when fixed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)