You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Beam JIRA Bot (Jira)" <ji...@apache.org> on 2020/09/26 17:08:02 UTC
[jira] [Updated] (BEAM-10516) Including TableProvider IO
dependencies as `provided` can lead to ClassDefNotFoundErrors for users
[ https://issues.apache.org/jira/browse/BEAM-10516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Beam JIRA Bot updated BEAM-10516:
---------------------------------
Labels: stale-assigned (was: )
> Including TableProvider IO dependencies as `provided` can lead to ClassDefNotFoundErrors for users
> --------------------------------------------------------------------------------------------------
>
> Key: BEAM-10516
> URL: https://issues.apache.org/jira/browse/BEAM-10516
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql, sdk-java-core
> Reporter: Scott Lukas
> Assignee: Brian Hulette
> Priority: P2
> Labels: stale-assigned
>
> The SQL extension uses ServiceLoader to load table providers: https://github.com/apache/beam/blob/v2.22.0/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamCalciteSchemaFactory.java#L85
> And the IOs used in the built-in table providers (e.g. pubsub json, avro, parquet, ...) are included as {{provided}} dependencies, not {{compile}}. This means that we are able to build the sql extension jar, and AutoService picks them up and records them in META-INF/services. However because they are {{provided}} dependencies, the necessary IO modules are not marked as dependencies for the SQL extension.
> This means consumers of :sdks:java:extensions:sql (like :sdks:java:extensions:sql:jdbc) can end up with ServiceLoader registering a TableProvider that will fail with ClassDefNotFoundError, unless they add the necessary dependency.
> Possible solution: Refactor TableProvider to beam core (or another shared module) in order to solve Service Loader errors with SchemaCapableIOTableProviderWrapper. Remove getTableType overrides of SchemaCapableIOProviderWrapper when fixed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)