You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Taylor Jones <mo...@gmail.com> on 2014/10/19 22:27:57 UTC
Drillbit hostname configuration error
Hi all,
I'm new to Drill, and I'm still trying to get a handle on how all of the
configuration options work. I was able to successfully follow the tutorial
for the embedded setup and I'm trying to get a distributed system set up
now. I thought I should take baby steps, so my environment is very simple:
- Host A: Zookeeper
- Host B: Drill
- Host C: Squirrel SQL + JDBC driver
I'm not using Hadoop yet, since there's only one Drill service running. I
followed all of the instructions, and I was able to successfully connect to
the Drill server using sqlline from Host B. However, when I try to connect
using Squirrel SQL from a remote server (Host C), I get an error.
Using connection URL "jdbc:drill:zk=HostA:2181/drill/drill-cluster" (my
cluster id is "drill-cluster") I get a large stacktrace, the root cause of
which is this:
Caused by: java.util.concurrent.ExecutionException:
java.net.ConnectException: Connection refused: localhost/127.0.0.1:31010
at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:37)
at
org.apache.drill.exec.rpc.BasicClient$ConnectionMultiListener$ConnectionHandler.operationComplete(BasicClient.java:154)
... 13 more
Caused by: java.net.ConnectException: Connection refused: localhost/
127.0.0.1:31010
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
at
io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:208)
at
io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:287)
... 6 more
Judging from the stacktrace, the JDBC driver is getting told to try to
connect to "localhost" instead of Host B. Thinking this odd, I went into
Zookeeper using zkCli.sh to look at the persisted config. Lo and behold,
"get /drill/drill-cluster/<HostB_znode>" is showing "localhost" as well:
[zk: localhost:2181(CONNECTED) 0] get
/drill/drill-cluster/48cc0eac-9be5-41a7-a530-ff0626dfce75
$48cc0eac-9be5-41a7-a530-ff0626dfce75���ϒ)
localhost���� ��
cZxid = 0x7f
ctime = Sun Oct 19 19:50:28 UTC 2014
mZxid = 0x7f
mtime = Sun Oct 19 19:50:28 UTC 2014
pZxid = 0x7f
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x14929cfe9210023
dataLength = 70
numChildren = 0
At this point, I'm fairly sure I this is my problem. It explains why Host B
can connect over sqlline (drillbit is available at localhost) and why I
can't connect from Host C (no drillbit at localhost). I'm still a little
unclear on how the client connection to Drillbits actually works, but I
assume that at some point an external client can connect to it. Judging
from the architecture diagrams, I believe Zookeeper returns the hostname/IP
address of one of the Drillbits to the JDBC driver.
So then, here's my question: How do I go about making the drillbit running
on Host B register itself in Zookeeper with its proper hostname instead of
"localhost"? Please keep in mind I'm new to Drill, so if I've grossly
misunderstood something please enlighten me :)
Re: Drillbit hostname configuration error
Posted by Taylor Jones <mo...@gmail.com>.
For posterity, I found that putting a line in /etc/hosts like this did the
trick:
<internal-ip> <external-domain-name>
I think the Drillbit is defaulting to doing an nslookup on the host's IP if
it doesn't have some record of it locally. Is that accurate?
On Sun, Oct 19, 2014 at 6:41 PM, Taylor Jones <mo...@gmail.com> wrote:
> Thanks for the advice, that certainly moved things along. Unfortunately
> I'm stuck again; I'm doing this on EC2, and now its using the
> internally-accessible hostname instead of the external one. The VM's
> hostname is set to the externally reachable one, could you explain how the
> Drillbit acquires its hostname? Also, is there any way to simply specify a
> hostname to use in the Drillbit config? I think in Hadoop a lot of this
> stuff is just automagically taken care of, but I'd like to understand more
> about how Drill works
>
> On Sun, Oct 19, 2014 at 5:20 PM, Aditya <ad...@gmail.com> wrote:
>
>> Could you check the "/etc/hosts" on the machine B and make sure that the
>> IP
>> "127.0.0.1" does not map to the hostname or the fqdn? It should look like
>>
>> 127.0.0.1 localhost.localdomain localhost
>>
>> and not
>>
>> 127.0.0.1 <host-b>.<domain> <host-b>
>>
>>
>> On Sun, Oct 19, 2014 at 1:27 PM, Taylor Jones <mo...@gmail.com>
>> wrote:
>>
>> > Hi all,
>> >
>> > I'm new to Drill, and I'm still trying to get a handle on how all of the
>> > configuration options work. I was able to successfully follow the
>> tutorial
>> > for the embedded setup and I'm trying to get a distributed system set up
>> > now. I thought I should take baby steps, so my environment is very
>> simple:
>> >
>> > - Host A: Zookeeper
>> > - Host B: Drill
>> > - Host C: Squirrel SQL + JDBC driver
>> >
>> > I'm not using Hadoop yet, since there's only one Drill service running.
>> I
>> > followed all of the instructions, and I was able to successfully
>> connect to
>> > the Drill server using sqlline from Host B. However, when I try to
>> connect
>> > using Squirrel SQL from a remote server (Host C), I get an error.
>> >
>> > Using connection URL "jdbc:drill:zk=HostA:2181/drill/drill-cluster" (my
>> > cluster id is "drill-cluster") I get a large stacktrace, the root cause
>> of
>> > which is this:
>> >
>> > Caused by: java.util.concurrent.ExecutionException:
>> > java.net.ConnectException: Connection refused: localhost/
>> 127.0.0.1:31010
>> > at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:37)
>> > at
>> >
>> >
>> org.apache.drill.exec.rpc.BasicClient$ConnectionMultiListener$ConnectionHandler.operationComplete(BasicClient.java:154)
>> > ... 13 more
>> > Caused by: java.net.ConnectException: Connection refused: localhost/
>> > 127.0.0.1:31010
>> > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>> > at
>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
>> > at
>> >
>> >
>> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:208)
>> > at
>> >
>> >
>> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:287)
>> > ... 6 more
>> >
>> > Judging from the stacktrace, the JDBC driver is getting told to try to
>> > connect to "localhost" instead of Host B. Thinking this odd, I went into
>> > Zookeeper using zkCli.sh to look at the persisted config. Lo and behold,
>> > "get /drill/drill-cluster/<HostB_znode>" is showing "localhost" as well:
>> >
>> > [zk: localhost:2181(CONNECTED) 0] get
>> > /drill/drill-cluster/48cc0eac-9be5-41a7-a530-ff0626dfce75
>> >
>> > $48cc0eac-9be5-41a7-a530-ff0626dfce75���ϒ)
>> > localhost���� ��
>> > cZxid = 0x7f
>> > ctime = Sun Oct 19 19:50:28 UTC 2014
>> > mZxid = 0x7f
>> > mtime = Sun Oct 19 19:50:28 UTC 2014
>> > pZxid = 0x7f
>> > cversion = 0
>> > dataVersion = 0
>> > aclVersion = 0
>> > ephemeralOwner = 0x14929cfe9210023
>> > dataLength = 70
>> > numChildren = 0
>> >
>> > At this point, I'm fairly sure I this is my problem. It explains why
>> Host B
>> > can connect over sqlline (drillbit is available at localhost) and why I
>> > can't connect from Host C (no drillbit at localhost). I'm still a little
>> > unclear on how the client connection to Drillbits actually works, but I
>> > assume that at some point an external client can connect to it. Judging
>> > from the architecture diagrams, I believe Zookeeper returns the
>> hostname/IP
>> > address of one of the Drillbits to the JDBC driver.
>> >
>> > So then, here's my question: How do I go about making the drillbit
>> running
>> > on Host B register itself in Zookeeper with its proper hostname instead
>> of
>> > "localhost"? Please keep in mind I'm new to Drill, so if I've grossly
>> > misunderstood something please enlighten me :)
>> >
>>
>
>
Re: Drillbit hostname configuration error
Posted by Taylor Jones <mo...@gmail.com>.
Thanks for the advice, that certainly moved things along. Unfortunately I'm
stuck again; I'm doing this on EC2, and now its using the
internally-accessible hostname instead of the external one. The VM's
hostname is set to the externally reachable one, could you explain how the
Drillbit acquires its hostname? Also, is there any way to simply specify a
hostname to use in the Drillbit config? I think in Hadoop a lot of this
stuff is just automagically taken care of, but I'd like to understand more
about how Drill works
On Sun, Oct 19, 2014 at 5:20 PM, Aditya <ad...@gmail.com> wrote:
> Could you check the "/etc/hosts" on the machine B and make sure that the IP
> "127.0.0.1" does not map to the hostname or the fqdn? It should look like
>
> 127.0.0.1 localhost.localdomain localhost
>
> and not
>
> 127.0.0.1 <host-b>.<domain> <host-b>
>
>
> On Sun, Oct 19, 2014 at 1:27 PM, Taylor Jones <mo...@gmail.com>
> wrote:
>
> > Hi all,
> >
> > I'm new to Drill, and I'm still trying to get a handle on how all of the
> > configuration options work. I was able to successfully follow the
> tutorial
> > for the embedded setup and I'm trying to get a distributed system set up
> > now. I thought I should take baby steps, so my environment is very
> simple:
> >
> > - Host A: Zookeeper
> > - Host B: Drill
> > - Host C: Squirrel SQL + JDBC driver
> >
> > I'm not using Hadoop yet, since there's only one Drill service running. I
> > followed all of the instructions, and I was able to successfully connect
> to
> > the Drill server using sqlline from Host B. However, when I try to
> connect
> > using Squirrel SQL from a remote server (Host C), I get an error.
> >
> > Using connection URL "jdbc:drill:zk=HostA:2181/drill/drill-cluster" (my
> > cluster id is "drill-cluster") I get a large stacktrace, the root cause
> of
> > which is this:
> >
> > Caused by: java.util.concurrent.ExecutionException:
> > java.net.ConnectException: Connection refused: localhost/127.0.0.1:31010
> > at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:37)
> > at
> >
> >
> org.apache.drill.exec.rpc.BasicClient$ConnectionMultiListener$ConnectionHandler.operationComplete(BasicClient.java:154)
> > ... 13 more
> > Caused by: java.net.ConnectException: Connection refused: localhost/
> > 127.0.0.1:31010
> > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
> > at
> >
> >
> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:208)
> > at
> >
> >
> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:287)
> > ... 6 more
> >
> > Judging from the stacktrace, the JDBC driver is getting told to try to
> > connect to "localhost" instead of Host B. Thinking this odd, I went into
> > Zookeeper using zkCli.sh to look at the persisted config. Lo and behold,
> > "get /drill/drill-cluster/<HostB_znode>" is showing "localhost" as well:
> >
> > [zk: localhost:2181(CONNECTED) 0] get
> > /drill/drill-cluster/48cc0eac-9be5-41a7-a530-ff0626dfce75
> >
> > $48cc0eac-9be5-41a7-a530-ff0626dfce75���ϒ)
> > localhost���� ��
> > cZxid = 0x7f
> > ctime = Sun Oct 19 19:50:28 UTC 2014
> > mZxid = 0x7f
> > mtime = Sun Oct 19 19:50:28 UTC 2014
> > pZxid = 0x7f
> > cversion = 0
> > dataVersion = 0
> > aclVersion = 0
> > ephemeralOwner = 0x14929cfe9210023
> > dataLength = 70
> > numChildren = 0
> >
> > At this point, I'm fairly sure I this is my problem. It explains why
> Host B
> > can connect over sqlline (drillbit is available at localhost) and why I
> > can't connect from Host C (no drillbit at localhost). I'm still a little
> > unclear on how the client connection to Drillbits actually works, but I
> > assume that at some point an external client can connect to it. Judging
> > from the architecture diagrams, I believe Zookeeper returns the
> hostname/IP
> > address of one of the Drillbits to the JDBC driver.
> >
> > So then, here's my question: How do I go about making the drillbit
> running
> > on Host B register itself in Zookeeper with its proper hostname instead
> of
> > "localhost"? Please keep in mind I'm new to Drill, so if I've grossly
> > misunderstood something please enlighten me :)
> >
>
Re: Drillbit hostname configuration error
Posted by Aditya <ad...@gmail.com>.
Could you check the "/etc/hosts" on the machine B and make sure that the IP
"127.0.0.1" does not map to the hostname or the fqdn? It should look like
127.0.0.1 localhost.localdomain localhost
and not
127.0.0.1 <host-b>.<domain> <host-b>
On Sun, Oct 19, 2014 at 1:27 PM, Taylor Jones <mo...@gmail.com> wrote:
> Hi all,
>
> I'm new to Drill, and I'm still trying to get a handle on how all of the
> configuration options work. I was able to successfully follow the tutorial
> for the embedded setup and I'm trying to get a distributed system set up
> now. I thought I should take baby steps, so my environment is very simple:
>
> - Host A: Zookeeper
> - Host B: Drill
> - Host C: Squirrel SQL + JDBC driver
>
> I'm not using Hadoop yet, since there's only one Drill service running. I
> followed all of the instructions, and I was able to successfully connect to
> the Drill server using sqlline from Host B. However, when I try to connect
> using Squirrel SQL from a remote server (Host C), I get an error.
>
> Using connection URL "jdbc:drill:zk=HostA:2181/drill/drill-cluster" (my
> cluster id is "drill-cluster") I get a large stacktrace, the root cause of
> which is this:
>
> Caused by: java.util.concurrent.ExecutionException:
> java.net.ConnectException: Connection refused: localhost/127.0.0.1:31010
> at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:37)
> at
>
> org.apache.drill.exec.rpc.BasicClient$ConnectionMultiListener$ConnectionHandler.operationComplete(BasicClient.java:154)
> ... 13 more
> Caused by: java.net.ConnectException: Connection refused: localhost/
> 127.0.0.1:31010
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
> at
>
> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:208)
> at
>
> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:287)
> ... 6 more
>
> Judging from the stacktrace, the JDBC driver is getting told to try to
> connect to "localhost" instead of Host B. Thinking this odd, I went into
> Zookeeper using zkCli.sh to look at the persisted config. Lo and behold,
> "get /drill/drill-cluster/<HostB_znode>" is showing "localhost" as well:
>
> [zk: localhost:2181(CONNECTED) 0] get
> /drill/drill-cluster/48cc0eac-9be5-41a7-a530-ff0626dfce75
>
> $48cc0eac-9be5-41a7-a530-ff0626dfce75���ϒ)
> localhost���� ��
> cZxid = 0x7f
> ctime = Sun Oct 19 19:50:28 UTC 2014
> mZxid = 0x7f
> mtime = Sun Oct 19 19:50:28 UTC 2014
> pZxid = 0x7f
> cversion = 0
> dataVersion = 0
> aclVersion = 0
> ephemeralOwner = 0x14929cfe9210023
> dataLength = 70
> numChildren = 0
>
> At this point, I'm fairly sure I this is my problem. It explains why Host B
> can connect over sqlline (drillbit is available at localhost) and why I
> can't connect from Host C (no drillbit at localhost). I'm still a little
> unclear on how the client connection to Drillbits actually works, but I
> assume that at some point an external client can connect to it. Judging
> from the architecture diagrams, I believe Zookeeper returns the hostname/IP
> address of one of the Drillbits to the JDBC driver.
>
> So then, here's my question: How do I go about making the drillbit running
> on Host B register itself in Zookeeper with its proper hostname instead of
> "localhost"? Please keep in mind I'm new to Drill, so if I've grossly
> misunderstood something please enlighten me :)
>