You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Vikas Balani (Jira)" <ji...@apache.org> on 2024/07/10 16:11:00 UTC
[jira] [Updated] (KAFKA-17111) ServiceConfigurationError in JsonSerializer/Deserializer during Plugin Discovery
[ https://issues.apache.org/jira/browse/KAFKA-17111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vikas Balani updated KAFKA-17111:
---------------------------------
Description:
h3. Problem:
JsonSerializer and JsonDeserializer use objectMapper.findAndRegisterModules(), which attempts to register all Jackson modules implementing com.fasterxml.jackson.databind.Module. This can cause a ServiceConfigurationError when incompatible modules are present in the classpath.
{code:java}
java.util.ServiceConfigurationError: org.apache.kafka.connect.storage.Converter: Provider org.apache.kafka.connect.json.JsonConverter could not be instantiated at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:586) at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:813) at java.base/java.util.ServiceLoader$ProviderImpl.get(ServiceLoader.java:729) at java.base/java.util.ServiceLoader$3.next(ServiceLoader.java:1403) at org.apache.kafka.connect.runtime.isolation.PluginScanner.handleLinkageError(PluginScanner.java:176) at org.apache.kafka.connect.runtime.isolation.PluginScanner.getServiceLoaderPluginDesc(PluginScanner.java:136) at org.apache.kafka.connect.runtime.isolation.ServiceLoaderScanner.scanPlugins(ServiceLoaderScanner.java:61) at org.apache.kafka.connect.runtime.isolation.PluginScanner.scanUrlsAndAddPlugins(PluginScanner.java:79) at org.apache.kafka.connect.runtime.isolation.PluginScanner.discoverPlugins(PluginScanner.java:67) at org.apache.kafka.connect.runtime.isolation.Plugins.initLoaders(Plugins.java:99) at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:90) at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:78) at org.apache.kafka.connect.cli.AbstractConnectCli.startConnect(AbstractConnectCli.java:128) at org.apache.kafka.connect.cli.AbstractConnectCli.run(AbstractConnectCli.java:101) at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:113) Caused by: java.util.ServiceConfigurationError: com.fasterxml.jackson.databind.Module: com.fasterxml.jackson.datatype.jsr310.JavaTimeModule not a subtype at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:593) at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1244) at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273) at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309) at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393) at com.fasterxml.jackson.databind.ObjectMapper.findModules(ObjectMapper.java:1158) at com.fasterxml.jackson.databind.ObjectMapper.findModules(ObjectMapper.java:1142) at com.fasterxml.jackson.databind.ObjectMapper.findAndRegisterModules(ObjectMapper.java:1192) at org.apache.kafka.connect.json.JsonSerializer.<init>(JsonSerializer.java:58) at org.apache.kafka.connect.json.JsonConverter.<init>(JsonConverter.java:250) at org.apache.kafka.connect.json.JsonConverter.<init>(JsonConverter.java:238) at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486) at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:789) ... 13 more{code}
h3. Steps to Reproduce:
1. Start a connector worker with Service loading enabled and with certain connector plugins in plugin path (e.g. AzureBlobSource & BigQuerySink)
2. Observe ServiceConfigurationError during plugin discovery
h3. Current Behavior:
ServiceConfigurationError is thrown with message "com.fasterxml.jackson.databind.Module: <module-name> not a subtype"
Where <module-name> can be one of: -
* com.fasterxml.jackson.module.jaxb.JaxbAnnotationModule
* com.fasterxml.jackson.datatype.jsr310.JavaTimeModule
* com.fasterxml.jackson.datatype.guava.GuavaModule
* com.fasterxml.jackson.datatype.joda.JodaModule
h3. Proposed Solution:
Explicitly register the Afterburner module instead of using findAndRegisterModules().
h3. Potential Impact:
- Resolves compatibility issues with certain Jackson modules
- Maintains performance improvements from Afterburner module
- May slightly change behavior for users relying on auto-registration of other Jackson modules
was:
h3. Problem:
JsonSerializer and JsonDeserializer use objectMapper.findAndRegisterModules(), which attempts to register all Jackson modules implementing com.fasterxml.jackson.databind.Module. This can cause a ServiceConfigurationError when incompatible modules are present in the classpath.
{code:java}
java.util.ServiceConfigurationError: org.apache.kafka.connect.storage.Converter: Provider org.apache.kafka.connect.json.JsonConverter could not be instantiated at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:586) at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:813) at java.base/java.util.ServiceLoader$ProviderImpl.get(ServiceLoader.java:729) at java.base/java.util.ServiceLoader$3.next(ServiceLoader.java:1403) at org.apache.kafka.connect.runtime.isolation.PluginScanner.handleLinkageError(PluginScanner.java:176) at org.apache.kafka.connect.runtime.isolation.PluginScanner.getServiceLoaderPluginDesc(PluginScanner.java:136) at org.apache.kafka.connect.runtime.isolation.ServiceLoaderScanner.scanPlugins(ServiceLoaderScanner.java:61) at org.apache.kafka.connect.runtime.isolation.PluginScanner.scanUrlsAndAddPlugins(PluginScanner.java:79) at org.apache.kafka.connect.runtime.isolation.PluginScanner.discoverPlugins(PluginScanner.java:67) at org.apache.kafka.connect.runtime.isolation.Plugins.initLoaders(Plugins.java:99) at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:90) at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:78) at org.apache.kafka.connect.cli.AbstractConnectCli.startConnect(AbstractConnectCli.java:128) at org.apache.kafka.connect.cli.AbstractConnectCli.run(AbstractConnectCli.java:101) at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:113) Caused by: java.util.ServiceConfigurationError: com.fasterxml.jackson.databind.Module: com.fasterxml.jackson.datatype.jsr310.JavaTimeModule not a subtype at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:593) at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1244) at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273) at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309) at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393) at com.fasterxml.jackson.databind.ObjectMapper.findModules(ObjectMapper.java:1158) at com.fasterxml.jackson.databind.ObjectMapper.findModules(ObjectMapper.java:1142) at com.fasterxml.jackson.databind.ObjectMapper.findAndRegisterModules(ObjectMapper.java:1192) at org.apache.kafka.connect.json.JsonSerializer.<init>(JsonSerializer.java:58) at org.apache.kafka.connect.json.JsonConverter.<init>(JsonConverter.java:250) at org.apache.kafka.connect.json.JsonConverter.<init>(JsonConverter.java:238) at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486) at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:789) ... 13 more{code}
h3. Steps to Reproduce:
1. Use JsonSerializer or JsonDeserializer with certain connector plugins (e.g. AzureBlobSource & BigQuerySink)
2. Observe ServiceConfigurationError during plugin discovery
h3. Current Behavior:
ServiceConfigurationError is thrown with message "com.fasterxml.jackson.databind.Module: <module-name> not a subtype"
Where <module-name> can be one of: -
* com.fasterxml.jackson.module.jaxb.JaxbAnnotationModule
* com.fasterxml.jackson.datatype.jsr310.JavaTimeModule
* com.fasterxml.jackson.datatype.guava.GuavaModule
* com.fasterxml.jackson.datatype.joda.JodaModule
h3. Proposed Solution:
Explicitly register the Afterburner module instead of using findAndRegisterModules().
h3. Potential Impact:
- Resolves compatibility issues with certain Jackson modules
- Maintains performance improvements from Afterburner module
- May slightly change behavior for users relying on auto-registration of other Jackson modules
> ServiceConfigurationError in JsonSerializer/Deserializer during Plugin Discovery
> --------------------------------------------------------------------------------
>
> Key: KAFKA-17111
> URL: https://issues.apache.org/jira/browse/KAFKA-17111
> Project: Kafka
> Issue Type: Bug
> Components: connect
> Affects Versions: 3.8.0
> Reporter: Vikas Balani
> Assignee: Vikas Balani
> Priority: Major
>
> h3. Problem:
> JsonSerializer and JsonDeserializer use objectMapper.findAndRegisterModules(), which attempts to register all Jackson modules implementing com.fasterxml.jackson.databind.Module. This can cause a ServiceConfigurationError when incompatible modules are present in the classpath.
>
> {code:java}
> java.util.ServiceConfigurationError: org.apache.kafka.connect.storage.Converter: Provider org.apache.kafka.connect.json.JsonConverter could not be instantiated at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:586) at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:813) at java.base/java.util.ServiceLoader$ProviderImpl.get(ServiceLoader.java:729) at java.base/java.util.ServiceLoader$3.next(ServiceLoader.java:1403) at org.apache.kafka.connect.runtime.isolation.PluginScanner.handleLinkageError(PluginScanner.java:176) at org.apache.kafka.connect.runtime.isolation.PluginScanner.getServiceLoaderPluginDesc(PluginScanner.java:136) at org.apache.kafka.connect.runtime.isolation.ServiceLoaderScanner.scanPlugins(ServiceLoaderScanner.java:61) at org.apache.kafka.connect.runtime.isolation.PluginScanner.scanUrlsAndAddPlugins(PluginScanner.java:79) at org.apache.kafka.connect.runtime.isolation.PluginScanner.discoverPlugins(PluginScanner.java:67) at org.apache.kafka.connect.runtime.isolation.Plugins.initLoaders(Plugins.java:99) at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:90) at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:78) at org.apache.kafka.connect.cli.AbstractConnectCli.startConnect(AbstractConnectCli.java:128) at org.apache.kafka.connect.cli.AbstractConnectCli.run(AbstractConnectCli.java:101) at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:113) Caused by: java.util.ServiceConfigurationError: com.fasterxml.jackson.databind.Module: com.fasterxml.jackson.datatype.jsr310.JavaTimeModule not a subtype at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:593) at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1244) at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273) at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309) at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393) at com.fasterxml.jackson.databind.ObjectMapper.findModules(ObjectMapper.java:1158) at com.fasterxml.jackson.databind.ObjectMapper.findModules(ObjectMapper.java:1142) at com.fasterxml.jackson.databind.ObjectMapper.findAndRegisterModules(ObjectMapper.java:1192) at org.apache.kafka.connect.json.JsonSerializer.<init>(JsonSerializer.java:58) at org.apache.kafka.connect.json.JsonConverter.<init>(JsonConverter.java:250) at org.apache.kafka.connect.json.JsonConverter.<init>(JsonConverter.java:238) at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486) at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:789) ... 13 more{code}
>
> h3. Steps to Reproduce:
> 1. Start a connector worker with Service loading enabled and with certain connector plugins in plugin path (e.g. AzureBlobSource & BigQuerySink)
> 2. Observe ServiceConfigurationError during plugin discovery
> h3. Current Behavior:
> ServiceConfigurationError is thrown with message "com.fasterxml.jackson.databind.Module: <module-name> not a subtype"
> Where <module-name> can be one of: -
> * com.fasterxml.jackson.module.jaxb.JaxbAnnotationModule
> * com.fasterxml.jackson.datatype.jsr310.JavaTimeModule
> * com.fasterxml.jackson.datatype.guava.GuavaModule
> * com.fasterxml.jackson.datatype.joda.JodaModule
> h3. Proposed Solution:
> Explicitly register the Afterburner module instead of using findAndRegisterModules().
> h3. Potential Impact:
> - Resolves compatibility issues with certain Jackson modules
> - Maintains performance improvements from Afterburner module
> - May slightly change behavior for users relying on auto-registration of other Jackson modules
--
This message was sent by Atlassian Jira
(v8.20.10#820010)