docker-compose example for Kafka Connect?

[Andrew Pennebaker <an...@gmail.com>]

I'm trying to get a basic Kafka Connect stack setup with docker-compose,
but the one offered by 1ambda,
https://hub.docker.com/r/1ambda/kafka-connect/ does not work. On my Mac,
the docker compose stack times out within a few dozen seconds.

Trace:

$ wget
https://raw.githubusercontent.com/1ambda/docker-kafka-connect/master/0.10.0.0/docker-compose.yml
$ docker-compose rm -f
$ docker-compose up
...
connect_1 | [2016-07-15 16:31:29,472] ERROR Uncaught exception in herder
work thread, exiting:
 (org.apache.kafka.connect.runtime.distributed.DistributedHerder:183)
connect_1 | org.apache.kafka.common.errors.TimeoutException: Timeout
expired while fetching topic metadata
connect_1 | [2016-07-15 16:31:29,477] INFO Kafka Connect stopping
(org.apache.kafka.connect.runtime.Connect:68)
connect_1 | [2016-07-15 16:31:29,477] INFO Stopping REST server
(org.apache.kafka.connect.runtime.rest.RestServer:154)
connect_1 | [2016-07-15 16:32:29,490] INFO Stopped ServerConnector@7418d711
{HTTP/1.1}{0.0.0.0:8083} (org.eclipse.jetty.server.ServerConnector:306)
connect_1 | [2016-07-15 16:32:29,511] INFO Stopped
o.e.j.s.ServletContextHandler@68fa0ba8{/,null,UNAVAILABLE}
(org.eclipse.jetty.server.handler.ContextHandler:865)
connect_1 | Exception in thread "Thread-1" java.lang.IllegalStateException:
!STOPPED
connect_1 | at
org.eclipse.jetty.server.handler.HandlerWrapper.destroy(HandlerWrapper.java:131)
connect_1 | at
org.apache.kafka.connect.runtime.rest.RestServer.stop(RestServer.java:162)
connect_1 | at
org.apache.kafka.connect.runtime.Connect.stop(Connect.java:70)
connect_1 | at
org.apache.kafka.connect.runtime.Connect$ShutdownHook.run(Connect.java:93)
connect_1 | [2016-07-15 16:32:50,578] WARN
 (org.eclipse.jetty.servlet.ServletHandler:620)
connect_1 | javax.servlet.ServletException: A MultiException has 1
exceptions.  They are:
connect_1 | 1. java.lang.IllegalStateException:
ServiceLocatorImpl(__HK2_Generated_0,1,844112759) has been shut down
connect_1 |
connect_1 | at
org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:489)
connect_1 | at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
connect_1 | at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
connect_1 | at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
connect_1 | at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
connect_1 | at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
connect_1 | at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
connect_1 | at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
connect_1 | at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
connect_1 | at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
connect_1 | at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
connect_1 | at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
connect_1 | at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
connect_1 | at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
connect_1 | at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
connect_1 | at
org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:159)
connect_1 | at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
connect_1 | at org.eclipse.jetty.server.Server.handle(Server.java:499)
connect_1 | at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
connect_1 | at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
connect_1 | at
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
connect_1 | at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
connect_1 | at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
connect_1 | at java.lang.Thread.run(Thread.java:745)
connect_1 | Caused by: A MultiException has 1 exceptions.  They are:
connect_1 | 1. java.lang.IllegalStateException:
ServiceLocatorImpl(__HK2_Generated_0,1,844112759) has been shut down
connect_1 |
connect_1 | at
org.jvnet.hk2.internal.FactoryCreator.getFactoryHandle(FactoryCreator.java:106)
connect_1 | at
org.jvnet.hk2.internal.FactoryCreator.dispose(FactoryCreator.java:173)
connect_1 | at
org.jvnet.hk2.internal.SystemDescriptor.dispose(SystemDescriptor.java:526)
connect_1 | at
org.glassfish.jersey.process.internal.RequestScope$Instance.remove(RequestScope.java:532)
connect_1 | at
org.glassfish.jersey.process.internal.RequestScope$Instance.release(RequestScope.java:549)
connect_1 | at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:319)
connect_1 | at
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
connect_1 | at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
connect_1 | at
org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
connect_1 | ... 23 more
connect_1 | Caused by: java.lang.IllegalStateException:
ServiceLocatorImpl(__HK2_Generated_0,1,844112759) has been shut down
connect_1 | at
org.jvnet.hk2.internal.ServiceLocatorImpl.checkState(ServiceLocatorImpl.java:2288)
connect_1 | at
org.jvnet.hk2.internal.ServiceLocatorImpl.getServiceHandleImpl(ServiceLocatorImpl.java:629)
connect_1 | at
org.jvnet.hk2.internal.ServiceLocatorImpl.getServiceHandle(ServiceLocatorImpl.java:622)
connect_1 | at
org.jvnet.hk2.internal.ServiceLocatorImpl.getServiceHandle(ServiceLocatorImpl.java:640)
connect_1 | at
org.jvnet.hk2.internal.FactoryCreator.getFactoryHandle(FactoryCreator.java:103)
connect_1 | ... 31 more
connect_1 | [2016-07-15 16:32:50,582] INFO 192.168.99.1 - - [1468600280533]
"GET /connectors HTTP/1.1" 500 294  90049
(org.apache.kafka.connect.runtime.rest.RestServer:60)
connect_1 | [2016-07-15 16:32:50,582] WARN /connectors
(org.eclipse.jetty.server.HttpChannel:396)
connect_1 | javax.servlet.ServletException: javax.servlet.ServletException:
A MultiException has 1 exceptions.  They are:
connect_1 | 1. java.lang.IllegalStateException:
ServiceLocatorImpl(__HK2_Generated_0,1,844112759) has been shut down
connect_1 |
connect_1 | at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:130)
connect_1 | at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
connect_1 | at
org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:159)
connect_1 | at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
connect_1 | at org.eclipse.jetty.server.Server.handle(Server.java:499)
connect_1 | at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
connect_1 | at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
connect_1 | at
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
connect_1 | at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
connect_1 | at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
connect_1 | at java.lang.Thread.run(Thread.java:745)
connect_1 | Caused by: javax.servlet.ServletException: A MultiException has
1 exceptions.  They are:
connect_1 | 1. java.lang.IllegalStateException:
ServiceLocatorImpl(__HK2_Generated_0,1,844112759) has been shut down
connect_1 |
connect_1 | at
org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:489)
connect_1 | at
org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
connect_1 | at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
connect_1 | at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
connect_1 | at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
connect_1 | at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
connect_1 | at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
connect_1 | at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
connect_1 | at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
connect_1 | at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
connect_1 | at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
connect_1 | at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
connect_1 | at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
connect_1 | at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
connect_1 | ... 10 more
connect_1 | Caused by: A MultiException has 1 exceptions.  They are:
connect_1 | 1. java.lang.IllegalStateException:
ServiceLocatorImpl(__HK2_Generated_0,1,844112759) has been shut down
connect_1 |
connect_1 | at
org.jvnet.hk2.internal.FactoryCreator.getFactoryHandle(FactoryCreator.java:106)
connect_1 | at
org.jvnet.hk2.internal.FactoryCreator.dispose(FactoryCreator.java:173)
connect_1 | at
org.jvnet.hk2.internal.SystemDescriptor.dispose(SystemDescriptor.java:526)
connect_1 | at
org.glassfish.jersey.process.internal.RequestScope$Instance.remove(RequestScope.java:532)
connect_1 | at
org.glassfish.jersey.process.internal.RequestScope$Instance.release(RequestScope.java:549)
connect_1 | at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:319)
connect_1 | at
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
connect_1 | at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
connect_1 | at
org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
connect_1 | ... 23 more
connect_1 | Caused by: java.lang.IllegalStateException:
ServiceLocatorImpl(__HK2_Generated_0,1,844112759) has been shut down
connect_1 | at
org.jvnet.hk2.internal.ServiceLocatorImpl.checkState(ServiceLocatorImpl.java:2288)
connect_1 | at
org.jvnet.hk2.internal.ServiceLocatorImpl.getServiceHandleImpl(ServiceLocatorImpl.java:629)
connect_1 | at
org.jvnet.hk2.internal.ServiceLocatorImpl.getServiceHandle(ServiceLocatorImpl.java:622)
connect_1 | at
org.jvnet.hk2.internal.ServiceLocatorImpl.getServiceHandle(ServiceLocatorImpl.java:640)
connect_1 | at
org.jvnet.hk2.internal.FactoryCreator.getFactoryHandle(FactoryCreator.java:103)
connect_1 | ... 31 more
connect_1 | [2016-07-15 16:32:50,584] WARN Could not send response error
500: javax.servlet.ServletException: javax.servlet.ServletException: A
MultiException has 1 exceptions.  They are:
connect_1 | 1. java.lang.IllegalStateException:
ServiceLocatorImpl(__HK2_Generated_0,1,844112759) has been shut down
connect_1 |  (org.eclipse.jetty.server.HttpChannel:482)
connect_1 | [2016-07-15 16:32:50,584] WARN FAILED
org.eclipse.jetty.server.Server@34408ed6:
java.util.concurrent.TimeoutException
(org.eclipse.jetty.util.component.AbstractLifeCycle:212)
connect_1 | java.util.concurrent.TimeoutException
connect_1 | at
org.eclipse.jetty.util.FutureCallback.get(FutureCallback.java:128)
connect_1 | at
org.eclipse.jetty.util.FutureCallback.get(FutureCallback.java:30)
connect_1 | at org.eclipse.jetty.server.Server.doStop(Server.java:426)
connect_1 | at
org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
connect_1 | at
org.apache.kafka.connect.runtime.rest.RestServer.stop(RestServer.java:157)
connect_1 | at
org.apache.kafka.connect.runtime.Connect.stop(Connect.java:70)
connect_1 | at
org.apache.kafka.connect.runtime.Connect$ShutdownHook.run(Connect.java:93)
hellokafkaconnect_connect_1 exited with code 0
Gracefully stopping... (press Ctrl+C again to force)
Stopping hellokafkaconnect_kafka_1... done
Stopping hellokafkaconnect_zk_1... done

Anyone know the cause of this error? Is there a more reliable
docker-compose setup than 1ambda's? Is Kafka 0.10.0.0 or Kafka Connect in
need of a bugfix?

-- 
Cheers,
Andrew