Ignite in Docker

[jimfcarroll <ji...@gmail.com>]

I know this question has been asked before but none of the solutions work for
us.

1) Running docker with --net=host seems to only work the clients are also
run this way. This isn't acceptable for our system given it's DC/OS (Mesos)
managed and elastic.
2) Running docker with an overlay network doesn't work in our distributed
DC/OS system since we rely on DC/OS for service discovery, etc.

3) I tried to use the AddressResolver Spi and I plugged it in in multiple
places but it seems to only 1/2 work. When I hit the rest service and get
the topology I only see the internal addresses. Multiple servers seem to
constantly cycle in and out of the cluster from the perspective on the
master (I'm assuming the first one started is the master) and "Topology
snapshot" messages are continuously pumped out showing servers=1, then
servers=2, then back to 1 over and over.

Clients behave the same way.

This seems to make the AddressResolver completely superfluous but I'm
assuming I'm doing something wrong.

Any guidance would be appreciated.

Thanks
Jim




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-in-Docker-tp10120.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite in Docker

[jimfcarroll <ji...@gmail.com>]

Hey, thanks. That was exactly the problem. The communication port was
obscured. 



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-in-Docker-tp10120p10185.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite in Docker

[vkulichenko <va...@gmail.com>]

Hi Jim,

If there are two nodes, but only one entry in the resolution map, how both
nodes provide proper mapping? Basically, it looks like discovery is working,
but communication is not. Note that both nodes must be able to instantiate
connection to each other.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-in-Docker-tp10120p10165.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite in Docker

[jimfcarroll <ji...@gmail.com>]

Sorry. I should also mention that we've overloaded discovery with an Spi. I
cut it from the config I posted to remove clutter. It works fine when we run
--net=host so I'm assuming that's not the problem.



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-in-Docker-tp10120p10128.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite in Docker

[jimfcarroll <ji...@gmail.com>]

So much for clarity. (1) should read "'resolution' is the name of the map I
mentioned." Sorry about that.



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-in-Docker-tp10120p10127.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite in Docker

[jimfcarroll <ji...@gmail.com>]

Two things to note if they're not obvious:

1) "resolver" is the name of the map I mentioned
2) My log messages show the mapping is being correctly done when I run it

Thanks again,
Jim




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-in-Docker-tp10120p10126.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite in Docker

[jimfcarroll <ji...@gmail.com>]

Hi Val,

Thanks for the response. The AddressResolver is pretty simple. It's
initialized with a map of internal to external IPs (currently the map has a
single entry). The the main getExternalAddress(final InetSocketAddress addr)
looks like this:

        try {
            LOGGER.debug("{} Looking up external address {}", ths, addr);

            String addrString = addr.getAddress().getHostAddress();
            if (addrString.startsWith("/")) {
                addrString = addrString.substring(1, addrString.length());
            }
            final String ret = resolution.get(addrString);

            LOGGER.debug("{} ... which resolved to {}", ths, ret);
 
            final Collection<InetSocketAddress> ret1 = Arrays.asList(ret ==
null ? addr : makeSocketAddr(addr, ret));
            if (ret != null) 
                LOGGER.debug("{}    ... final result {}", ths, ret1);
            return ret1;
        } catch (final UnknownHostException uhe) {
            throw new IgniteCheckedException(uhe);
        }

Note: 'ths' is an instance count sequence provided from a static
AtomicInteger for debug purposes only.

I have this plugged in using Spring (the class is called
DockerIgniteServerAddressResolver. Package names redacted):

  <bean id="grid.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">
    <property name="discoverySpi">
      <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
        <property name="addressResolver">
          <bean class="com....DockerIgniteServerAddressResolver" />
        </property>
      </bean>
    </property>
    <property name="communicationSpi">
      <bean
class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
        <property name="addressResolver">
          <bean class="com...DockerIgniteServerAddressResolver" />
        </property>
      </bean>
    </property>
    <property name="addressResolver">
      <bean class="com....DockerIgniteServerAddressResolver" />
    </property>
  </bean>
</beans>

I tried this plugging the resolver into different places. It wasn't
originally provided in all 3.

This is server configuration only. If I run 1 server and 1 client it seems
okay. If I run a 2nd server I get the following repeatedly in ONE of the
servers (and the client- if it's running):

[16:59:05] Topology snapshot [ver=24004, servers=2, clients=0, CPUs=8,
heap=7.1GB]
[16:59:05] Topology snapshot [ver=24005, servers=1, clients=0, CPUs=4,
heap=3.6GB]

This repeats every second or so.

If I do a curl host:port/ignite?cmd=top I see the INTERNAL addresses in
tcpAddresses. I followed through the code and can see this is the case, even
though the external addresses are available in the code that sets the bean
values, so I'm assuming that's the intention.

I'm running 1.8.0.

Thanks
Jim








--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-in-Docker-tp10120p10125.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite in Docker

[vkulichenko <va...@gmail.com>]

Hi Jim,

Please show you AddressResolver implementation, your configuration and
describe the deployment in more details (how many nodes, how addresses are
assigned to them, etc).

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-in-Docker-tp10120p10124.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.