You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by John Smith <ja...@gmail.com> on 2020/06/01 13:31:43 UTC
Re: Zookeeper discovery in mix environments.
Any news on this? Thanks
On Thu., May 28, 2020, 1:10 p.m. John Smith, <ja...@gmail.com> wrote:
> Hi, I'm running an ignite cluster on VMs running on Openstack and using
> regular network stack nothing special here.
>
> My CLIENT (client=true) applications are running in DC/OS using docker
> container in bridged network mode.
>
> When using TCP discovery everything works nice and dandy. But I recently
> tried to switch to Zookeeper discovery but the docker application can't
> seem to connect or get discovered?
>
> I'm assuming when running in bridge mode I need to open a port on the
> bridge to allow other node to connect back. Is there a way to the ignite
> client/zookeeper SPI that the CLIENT node is available at a specific
> ip/port?
>
Re: Zookeeper discovery in mix environments.
Posted by John Smith <ja...@gmail.com>.
We know that host mode works, that's too easy... And I said that already
works no problem.
So the client is running in Dokcer on bridge mode. It binds to some
internal docker I.P 172.18.xxx.xxx That can also be easily seen with Visor.
So in bridge mode it means we need to map some external I.P + port to the
internal and we need to say to the client that even though he bound to some
internal address he needs to present himself as some other I.P.
On Mon, 8 Jun 2020 at 19:03, akorensh <al...@gmail.com> wrote:
> Limit your env to two nodes.
> Try using --net=host for docker containers. If that works, limiting to
> use
> setLocalHost for TcpCommunicationSPI, and configure ZooKeeper to use
> address
> translation/or bind to a specific address.
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Zookeeper discovery in mix environments.
Posted by akorensh <al...@gmail.com>.
Limit your env to two nodes.
Try using --net=host for docker containers. If that works, limiting to use
setLocalHost for TcpCommunicationSPI, and configure ZooKeeper to use address
translation/or bind to a specific address.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Zookeeper discovery in mix environments.
Posted by John Smith <ja...@gmail.com>.
I think the problem might also, be the client is running inside DC/OS with
docker containers. When I check the network topology all
connected aplication are reporting multiple addresses...
Here is an example of a single client... Is it possible to tell it only
bind to one or only use one?
----------+
| xxx.xxx.0.17 | 1: xxxx(@n1) | xxxx | Client | Linux amd64
3.10.0-862.14.4.el7.x86_64 | 2 | xxxx | 1.00 % |
| xxx.xxxx.0.1 | |
| | | | xxxxxx |
|
| xxx.xxx.100.1 | |
| | | | xxxxxx |
|
| xxx.xxx.100.2 | |
| | | | xxxxxx |
|
| 127.0.0.1 | |
| | | | xxxxxx |
|
| xxx.xxx.xxx.68 | |
| | | | xxxxxx |
|
| xxx.xxx.0.1 | |
| | | |
| |
| xxx.xxx.xxx.129 | |
| | | |
| |
| xxx.xxx.100.3 | |
| | | |
| |
+--------------------+------------------+--------------------------------------+-----------+----------------------------------------+------+-------------------
On Mon, 8 Jun 2020 at 17:02, akorensh <al...@gmail.com> wrote:
> In terms of TcpCommunicationSPI it might help:
>
> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/spi/communication/tcp/TcpCommunicationSpi.html#setLocalAddress-java.lang.String-
>
> Also look into setting the localaddress:
>
> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/spi/communication/tcp/TcpCommunicationSpi.html#setLocalAddress-java.lang.String-
>
> Doc for tcp communication spi:
> https://apacheignite.readme.io/docs/network-config#tcpcommunicationspi
>
>
>
> In your case, Discovery Communcation is using ZooKeeper to exchange
> messages, and therefore you would need to configure ZooKeeper if you would
> like address resolution.
>
> Ignite class ZookeperClient is a wrapper around the native ZooKeeper class
> which handles all communication.
>
> ZookeperClient wrapper class:
>
> https://github.com/apache/ignite/blob/master/modules/zookeeper/src/main/java/org/apache/ignite/spi/discovery/zk/internal/ZookeeperClient.java
>
>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Zookeeper discovery in mix environments.
Posted by akorensh <al...@gmail.com>.
In terms of TcpCommunicationSPI it might help:
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/spi/communication/tcp/TcpCommunicationSpi.html#setLocalAddress-java.lang.String-
Also look into setting the localaddress:
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/spi/communication/tcp/TcpCommunicationSpi.html#setLocalAddress-java.lang.String-
Doc for tcp communication spi:
https://apacheignite.readme.io/docs/network-config#tcpcommunicationspi
In your case, Discovery Communcation is using ZooKeeper to exchange
messages, and therefore you would need to configure ZooKeeper if you would
like address resolution.
Ignite class ZookeperClient is a wrapper around the native ZooKeeper class
which handles all communication.
ZookeperClient wrapper class:
https://github.com/apache/ignite/blob/master/modules/zookeeper/src/main/java/org/apache/ignite/spi/discovery/zk/internal/ZookeeperClient.java
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Zookeeper discovery in mix environments.
Posted by John Smith <ja...@gmail.com>.
Would address resolver work here with Zookeeper discovery? I think most of
my communications erros are due to my clients binding to multiple addresses?
Re: Zookeeper discovery in mix environments.
Posted by akorensh <al...@gmail.com>.
It is possible. All discovery communication is through ZooKeeper, but yes
private docker IPs (172.. ) might be leaked over. Would need to see the full
logs(Ignite and Zookeper) which contain the IPs of the node and the
particular one used for a given communication. Try running docker
w/--net=host first to get everything working (take a look at the logs to see
what is happening in this scenario) and then go back to working w/the
bridged network.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Zookeeper discovery in mix environments.
Posted by John Smith <ja...@gmail.com>.
So, I'm guessing that the client node reports multiple I.Ps to the
zookeeper so then when another node tries to connect to the client node to
create the full mesh maybe it connecting to the wrong I.P?
On Mon, 1 Jun 2020 at 11:26, akorensh <al...@gmail.com> wrote:
> Hi,
> ZookeperDiscoverySpi
> <
> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/spi/discovery/zk/ZookeeperDiscoverySpi.html>
>
> uses ZooKeeper to transmit discovery messages.
>
> You can specify a comma separated host:port pairs, each corresponding to
> a
> zooKeeper server here:
> https://apacheignite.readme.io/docs/zookeeper-discovery#configuration
>
> Internally it utilizes the ZooKeeper Client API:
>
>
> https://zookeeper.apache.org/doc/r3.4.13/api/org/apache/zookeeper/ZooKeeper.html#ZooKeeper-java.lang.String-int-org.apache.zookeeper.Watcher-
>
> You need to open up the ports specified in your connection string in your
> bridged network.
>
> Thanks, Alex
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Zookeeper discovery in mix environments.
Posted by akorensh <al...@gmail.com>.
Hi,
ZookeperDiscoverySpi
<https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/spi/discovery/zk/ZookeeperDiscoverySpi.html>
uses ZooKeeper to transmit discovery messages.
You can specify a comma separated host:port pairs, each corresponding to a
zooKeeper server here:
https://apacheignite.readme.io/docs/zookeeper-discovery#configuration
Internally it utilizes the ZooKeeper Client API:
https://zookeeper.apache.org/doc/r3.4.13/api/org/apache/zookeeper/ZooKeeper.html#ZooKeeper-java.lang.String-int-org.apache.zookeeper.Watcher-
You need to open up the ports specified in your connection string in your
bridged network.
Thanks, Alex
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Zookeeper discovery in mix environments.
Posted by Wesley Peng <we...@gmail.com>.
Maybe you would ask the question on VM/container related mailing list?
Regards
On Mon, Jun 1, 2020 at 9:32 PM John Smith <ja...@gmail.com> wrote:
> Any news on this? Thanks
>
>
> On Thu., May 28, 2020, 1:10 p.m. John Smith, <ja...@gmail.com>
> wrote:
>
>> Hi, I'm running an ignite cluster on VMs running on Openstack and using
>> regular network stack nothing special here.
>>
>> My CLIENT (client=true) applications are running in DC/OS using docker
>> container in bridged network mode.
>>
>> When using TCP discovery everything works nice and dandy. But I recently
>> tried to switch to Zookeeper discovery but the docker application can't
>> seem to connect or get discovered?
>>
>> I'm assuming when running in bridge mode I need to open a port on the
>> bridge to allow other node to connect back. Is there a way to the ignite
>> client/zookeeper SPI that the CLIENT node is available at a specific
>> ip/port?
>>
>