You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Ewen Cheslack-Postava (JIRA)" <ji...@apache.org> on 2016/05/10 23:54:12 UTC
[jira] [Commented] (KAFKA-3690) Avoid passing null to
UnmodifiableMap
[ https://issues.apache.org/jira/browse/KAFKA-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15279257#comment-15279257 ]
Ewen Cheslack-Postava commented on KAFKA-3690:
----------------------------------------------
Patch available here https://github.com/apache/kafka/pull/1360 thanks to [~liquanpei]. I want to discuss whether this is actually a blocker. The patch reveals that the issue is a bug in SchemaBuilder combined with a misuse of SchemaBuilder without having called build() to get the final immutable Schema out of it. However, as far as I can tell we will only currently hit this case if both a) SchemaBuilder is left "uncompiled" incorrectly and b) a Converter triggers the issue by calling Schema.parameters(). Right now we're seeing that triggered because KafkaConfigBackingStore (in previous release and current RC) are not compiling the schema, but only seeing it for this user because they are using JsonConverter with schemas *enabled*, which is *not* how we provide example configs although is (perhaps unfortunately) the default for JsonConverter. Someone using AvroConverter as the internal converter would hit the same issue as it *always* has to convert schemas and will always call that method.
I'm unsure of the exact impact here. The details of the internal schema management aren't *really* of concern to the user. Is advising users to use JsonConverter with flags to turn off the schema envelope a good enough solution? Most users wouldn't even think about swapping out the internal converters anyway, I suspect someone happened upon this issue just by paring down existing configs or trying to write a worker config entirely from scratch. Certainly for this user, who is on a previous version (not 0.10 RC), that will be the only solution unless they upgrade.
> Avoid passing null to UnmodifiableMap
> -------------------------------------
>
> Key: KAFKA-3690
> URL: https://issues.apache.org/jira/browse/KAFKA-3690
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 0.10.0.0
> Reporter: Liquan Pei
> Assignee: Liquan Pei
> Fix For: 0.10.0.0
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> For some version of JDK, if we pass null to UnModifiableMap's constructor, an NullPointerException will be thrown.
> This is produced by
> curl -X POST -H "Content-Type: application/json" --data '{"name": "local-conscq!-source", "config": {"connector.class":"org.apache.kafka.connect.file.FileStreamSourceConnector", "tasks.max":"1", "topic":"connect-test" }}' http://localhost:8083/connectors
> 2016-05-09 12:36:03,974] INFO Starting connectors and tasks using config offset -1 (org.apache.kafka.connect.runtime.distributed.DistributedHerder:639)
> [2016-05-09 12:36:03,974] INFO Finished starting connectors and tasks (org.apache.kafka.connect.runtime.distributed.DistributedHerder:659)
> [2016-05-09 12:36:42,691] WARN (org.eclipse.jetty.servlet.ServletHandler:620)
> javax.servlet.ServletException: java.lang.NullPointerException
> at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:487)
> at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:425)
> at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:383)
> at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:336)
> at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:223)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:808)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:159)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at java.util.Collections$UnmodifiableMap.<init>(Collections.java:1446)
> at java.util.Collections.unmodifiableMap(Collections.java:1433)
> at org.apache.kafka.connect.data.SchemaBuilder.parameters(SchemaBuilder.java:184)
> at org.apache.kafka.connect.json.JsonConverter.asJsonSchema(JsonConverter.java:415)
> at org.apache.kafka.connect.json.JsonConverter.asJsonSchema(JsonConverter.java:398)
> at org.apache.kafka.connect.json.JsonConverter.convertToJsonWithEnvelope(JsonConverter.java:551)
> at org.apache.kafka.connect.json.JsonConverter.fromConnectData(JsonConverter.java:305)
> at org.apache.kafka.connect.storage.KafkaConfigStorage.putConnectorConfig(KafkaConfigStorage.java:282)
> at org.apache.kafka.connect.runtime.distributed.DistributedHerder$4.call(DistributedHerder.java:428)
> at org.apache.kafka.connect.runtime.distributed.DistributedHerder$4.call(DistributedHerder.java:407)
> at org.apache.kafka.connect.runtime.distributed.DistributedHerder.tick(DistributedHerder.java:210)
> at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:159)
> ... 1 more
> [2016-05-09 12:36:42,694] INFO 127.0.0.1 - - [09/May/2016:19:36:42 +0000] "POST /connectors HTTP/1.1" 500 294 133 (org.apache.kafka.connect.runtime.rest.RestServer:60)
> [2016-05-09 12:36:42,694] WARN /connectors (org.eclipse.jetty.server.HttpChannel:395)
> javax.servlet.ServletException: javax.servlet.ServletException: java.lang.NullPointerException
> at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:130)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:159)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: javax.servlet.ServletException: java.lang.NullPointerException
> at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:487)
> at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:425)
> at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:383)
> at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:336)
> at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:223)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:808)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> ... 10 more
> Caused by: java.lang.NullPointerException
> at java.util.Collections$UnmodifiableMap.<init>(Collections.java:1446)
> at java.util.Collections.unmodifiableMap(Collections.java:1433)
> at org.apache.kafka.connect.data.SchemaBuilder.parameters(SchemaBuilder.java:184)
> at org.apache.kafka.connect.json.JsonConverter.asJsonSchema(JsonConverter.java:415)
> at org.apache.kafka.connect.json.JsonConverter.asJsonSchema(JsonConverter.java:398)
> at org.apache.kafka.connect.json.JsonConverter.convertToJsonWithEnvelope(JsonConverter.java:551)
> at org.apache.kafka.connect.json.JsonConverter.fromConnectData(JsonConverter.java:305)
> at org.apache.kafka.connect.storage.KafkaConfigStorage.putConnectorConfig(KafkaConfigStorage.java:282)
> at org.apache.kafka.connect.runtime.distributed.DistributedHerder$4.call(DistributedHerder.java:428)
> at org.apache.kafka.connect.runtime.distributed.DistributedHerder$4.call(DistributedHerder.java:407)
> at org.apache.kafka.connect.runtime.distributed.DistributedHerder.tick(DistributedHerder.java:210)
> at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:159)
> ... 1 more
> [2016-05-09 12:36:42,696] WARN Could not send response error 500: javax.servlet.ServletException: javax.servlet.ServletException: java.lang.NullPointerException (org.eclipse.jetty.server.HttpChannel:481)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)