You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2019/07/25 16:25:12 UTC
[GitHub] [pulsar] sudeepamin opened a new issue #4812: Pulsar cannot resolve
AWS ELB URL
sudeepamin opened a new issue #4812: Pulsar cannot resolve AWS ELB URL
URL: https://github.com/apache/pulsar/issues/4812
**Describe the bug**
Pulsar libraries cannot connect to Pulsar via a Load Balancer URL setup as a part of the 'streamlio-admin ingress create' command. If you directly use one of the IP addresses that the URL resolves to the code is able to connect to Pulsar. However, AWS recommends against the use of individual IP addresses directly and to use the ELB LoadBalancer URL.
**To Reproduce**
Steps to reproduce the behavior:
**Java code:**
try {
client = PulsarClient.builder()
.serviceUrl("pulsar://a5b327ee3ad6d11e989f80a0e47fe403-954928901.us-east-1.elb.amazonaws.com:6650")
.build();
}
**Error:**
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'log4j2.debug' to show Log4j2 internal initialization logging.
org.apache.pulsar.client.api.PulsarClientException: java.util.concurrent.CompletionException: org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext$SearchDomainUnknownHostException: Search domain query failed. Original hostname: 'a5b327ee3ad6d11e989f80a0e47fe403-954928901.us-east-1.elb.amazonaws.com' failed to resolve 'a5b327ee3ad6d11e989f80a0e47fe403-954928901.us-east-1.elb.amazonaws.com' after 6 queries
at org.apache.pulsar.client.impl.ConnectionPool.lambda$null$9(ConnectionPool.java:202)
at org.apache.pulsar.shade.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at org.apache.pulsar.shade.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at org.apache.pulsar.shade.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:335)
at org.apache.pulsar.shade.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909)
at org.apache.pulsar.shade.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.util.concurrent.CompletionException: org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext$SearchDomainUnknownHostException: Search domain query failed. Original hostname: 'a5b327ee3ad6d11e989f80a0e47fe403-954928901.us-east-1.elb.amazonaws.com' failed to resolve 'a5b327ee3ad6d11e989f80a0e47fe403-954928901.us-east-1.elb.amazonaws.com' after 6 queries
at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331)
at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:346)
at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1063)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
at org.apache.pulsar.client.impl.ConnectionPool.lambda$resolveName$16(ConnectionPool.java:259)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:511)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:485)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:424)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:121)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext$2.operationComplete(DnsResolveContext.java:198)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:511)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:485)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:424)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:121)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext.finishResolve(DnsResolveContext.java:856)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(DnsResolveContext.java:809)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext.query(DnsResolveContext.java:332)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext.access$600(DnsResolveContext.java:62)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext$3.operationComplete(DnsResolveContext.java:381)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:511)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:485)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:424)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:162)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:33)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext.query(DnsResolveContext.java:356)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext.access$600(DnsResolveContext.java:62)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext$3.operationComplete(DnsResolveContext.java:381)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:511)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:504)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:483)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:424)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:121)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsQueryContext.setFailure(DnsQueryContext.java:220)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsQueryContext.access$300(DnsQueryContext.java:43)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsQueryContext$4.run(DnsQueryContext.java:170)
at org.apache.pulsar.shade.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
at org.apache.pulsar.shade.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127)
... 6 more
Caused by: org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext$SearchDomainUnknownHostException: Search domain query failed. Original hostname: 'a5b327ee3ad6d11e989f80a0e47fe403-954928901.us-east-1.elb.amazonaws.com' failed to resolve 'a5b327ee3ad6d11e989f80a0e47fe403-954928901.us-east-1.elb.amazonaws.com' after 6 queries
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext.finishResolve(DnsResolveContext.java:848)
... 28 more
Caused by: org.apache.pulsar.shade.io.netty.resolver.dns.DnsNameResolverException: [/fec0:0:0:ffff:0:0:0:1:53] failed to send a query (no stack trace available)
Caused by: org.apache.pulsar.shade.io.netty.channel.unix.Errors$NativeIoException: sendToAddress(..) failed: Network is unreachable
at org.apache.pulsar.shade.io.netty.channel.unix.Errors.newIOException(Errors.java:122)
at org.apache.pulsar.shade.io.netty.channel.unix.Errors.ioResult(Errors.java:146)
at org.apache.pulsar.shade.io.netty.channel.unix.Socket.sendToAddress(Socket.java:175)
at org.apache.pulsar.shade.io.netty.channel.epoll.EpollDatagramChannel.doWriteMessage(EpollDatagramChannel.java:347)
at org.apache.pulsar.shade.io.netty.channel.epoll.EpollDatagramChannel.doWrite(EpollDatagramChannel.java:300)
at org.apache.pulsar.shade.io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:938)
at org.apache.pulsar.shade.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.flush0(AbstractEpollChannel.java:512)
at org.apache.pulsar.shade.io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:905)
at org.apache.pulsar.shade.io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1396)
at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776)
at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:768)
at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:749)
at org.apache.pulsar.shade.io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776)
at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:802)
at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:814)
at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:794)
at org.apache.pulsar.shade.io.netty.channel.DefaultChannelPipeline.writeAndFlush(DefaultChannelPipeline.java:1066)
at org.apache.pulsar.shade.io.netty.channel.AbstractChannel.writeAndFlush(AbstractChannel.java:309)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsQueryContext.writeQuery(DnsQueryContext.java:139)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsQueryContext.sendQuery(DnsQueryContext.java:118)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsQueryContext.query(DnsQueryContext.java:113)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsNameResolver.query0(DnsNameResolver.java:1109)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext.query(DnsResolveContext.java:350)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext.access$600(DnsResolveContext.java:62)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext$3.operationComplete(DnsResolveContext.java:381)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:511)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:504)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:483)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:424)
at org.apache.pulsar.shade.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:121)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsQueryContext.setFailure(DnsQueryContext.java:220)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsQueryContext.access$300(DnsQueryContext.java:43)
at org.apache.pulsar.shade.io.netty.resolver.dns.DnsQueryContext$4.run(DnsQueryContext.java:170)
at org.apache.pulsar.shade.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
at org.apache.pulsar.shade.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127)
at org.apache.pulsar.shade.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at org.apache.pulsar.shade.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at org.apache.pulsar.shade.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:335)
at org.apache.pulsar.shade.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909)
at org.apache.pulsar.shade.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
nslookup on the LoadBalancer URL gives the following:
nslookup ab30d3c79aee611e9acf102a8b3fc243-31673255.us-east-1.elb.amazonaws.com
Server: 172.31.0.2
Address: 172.31.0.2#53
Non-authoritative answer:
Name: ab30d3c79aee611e9acf102a8b3fc243-31673255.us-east-1.elb.amazonaws.com
Address: 23.23.22.177
Name: ab30d3c79aee611e9acf102a8b3fc243-31673255.us-east-1.elb.amazonaws.com
Address: 34.199.105.217
Name: ab30d3c79aee611e9acf102a8b3fc243-31673255.us-east-1.elb.amazonaws.com
Address: 3.210.69.95
**Desktop (please complete the following information):**
- OS: Ubuntu 18.04
**Additional context**
Looks like the issue is with 'org.apache.pulsar.shade.io.netty.resolver.dns.DnsResolveContext$' not being able to handle multiple URLs when the LoadBalancer IP address resolves to multiple addresses. When the code uses individual IP addresses the code works without issues. But as mentioned above AWS recommends against using individual addresses directly as it defeats the purpose of loadbalancing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services