You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by relax ken <re...@gmail.com> on 2019/04/16 16:45:35 UTC
Node joining didn't trigger Affinity recalculation problem
Hi,
I wrote an aws ELB based tcp discovery ip finder. When I launched two ec2
nodes in the same target group, the ip finder found node IPs correctly.
However, affinity recalculation wasn't triggered. Basically nothing
happened. also no exception. If I ran two nodes locally with multicast,
affinity recalculation worked properly. I just wonder what else I need
to do to make the custom IP finder works?
Here is the overrided method in IPFinder. Basically it just is a slightly
modified version of TcpDiscoveryElbIpFinder
@Override public Collection<InetSocketAddress>
getRegisteredAddresses() throws IgniteSpiException {
initClients();
logger.debug("elb host discovery");
List<String> privateIPs =
awsService.getHealthyPrivateIPsFromTarget(targetGroupArn);
List<InetSocketAddress> addrs = new ArrayList<>();
for (String privateIP : privateIPs) {
logger.debug("tcp discovery. found host "+privateIP);
addrs.add(new InetSocketAddress(privateIP, 0));
}
return addrs;
}
@Override public void registerAddresses(Collection<InetSocketAddress>
addrs) throws IgniteSpiException {
//No-op, ELB will take care of registration.
}
@Override public void
unregisterAddresses(Collection<InetSocketAddress> addrs) throws
IgniteSpiException {
// No-op, ELB will take care of this process.
}
And here is the ignite config setting
TcpDiscoverySpi spi = new TcpDiscoverySpi();
spi.setIpFinder(getTcpELBIpFinder(elbDiscoveryConfig));
IgniteConfiguration igniteConfig = new IgniteConfiguration();
igniteConfig.setMetricsLogFrequency(0);
igniteConfig.setDiscoverySpi(spi);
DataStorageConfiguration storageCfg = new DataStorageConfiguration();
storageCfg.getDefaultDataRegionConfiguration().setPersistenceEnabled(false);
igniteConfig.setDataStorageConfiguration(storageCfg);
Any ideas?
Thanks
Ken
Re: Node joining didn't trigger Affinity recalculation problem
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I guess this is because you can't join two clusters together. Once each
node has formed a cluster at start, neither one can just other cluster.
A cluster can accept nodes, new nodes can join to cluster, but two clusters
can't merge. The only way here is to restart one of nodes.
Regards,
--
Ilya Kasnacheev
чт, 25 апр. 2019 г. в 16:17, relax ken <re...@gmail.com>:
> Hi Denis,
>
> I just found an issue with the ec2 tcp discovery. If I have one node
> running in elb and launch a new node in elb, they will join the same
> cluster properly. However, if I launch two nodes separately first without
> joining elb and then add them into the same elb after they are fully
> running, they will find each other by ip finder but won't join the cluster.
> Any ideas?
>
> Thanks
>
> On Sat, Apr 20, 2019 at 7:01 AM Denis Magda <dm...@apache.org> wrote:
>
>> No problem, thanks for taking the time and sharing the issue. We'll
>> update the docs. Good luck with your Ignite deployment!
>>
>> -
>> Denis
>>
>>
>> On Fri, Apr 19, 2019 at 10:32 PM relax ken <re...@gmail.com> wrote:
>>
>>> Great! Thanks Denis! that's the issue. After I configured it in security
>>> group, it works
>>>
>>> On Sat, Apr 20, 2019 at 6:03 AM Denis Magda <dm...@apache.org> wrote:
>>>
>>>> By default, each node will be connecting to port 47500 and the next 100
>>>> ports if nobody responds on 45700, 457001, etc.
>>>>
>>>> Please check that the port is opened. Alternatively, you can always
>>>> manage the port with TcpDiscoverySpi.setLocalPort [1]. The ports range can
>>>> be adjusted as well. [2]
>>>>
>>>> [1]
>>>> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/TcpDiscoverySpi.java#L771
>>>> [2]
>>>> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/TcpDiscoverySpi.java#L782
>>>>
>>>> -
>>>> Denis
>>>>
>>>>
>>>> On Fri, Apr 19, 2019 at 9:38 PM relax ken <re...@gmail.com> wrote:
>>>>
>>>>> I don't think both nodes joined the cluster. That is the problem. I
>>>>> don't know what it's wrong and why they are not in the cluster. They are in
>>>>> the same subnet,. They can talk to each other. They found each other's IP.
>>>>>
>>>>> What port does ignite use for cluster joining? It may be security
>>>>> group issue. Will ignite throw exception or log warning if they can't join
>>>>> the cluster?
>>>>>
>>>>> Thanks
>>>>>
>>>>> On Sat, 20 Apr 2019, 05:50 Denis Magda, <dm...@apache.org> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Could you confirm that both nodes on AWS joined the cluster? Data
>>>>>> will be rebalanced between the nodes once you see 2 server nodes in your
>>>>>> cluster topology. Might be some networking issue.
>>>>>>
>>>>>> -
>>>>>> Denis
>>>>>>
>>>>>>
>>>>>> On Tue, Apr 16, 2019 at 9:53 AM relax ken <re...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I wrote an aws ELB based tcp discovery ip finder. When I launched
>>>>>>> two ec2 nodes in the same target group, the ip finder found node IPs
>>>>>>> correctly. However, affinity recalculation wasn't triggered. Basically
>>>>>>> nothing happened. also no exception. If I ran two nodes locally with
>>>>>>> multicast, affinity recalculation worked properly. I just wonder what else
>>>>>>> I need to do to make the custom IP finder works?
>>>>>>>
>>>>>>> Here is the overrided method in IPFinder. Basically it just is a
>>>>>>> slightly modified version of TcpDiscoveryElbIpFinder
>>>>>>>
>>>>>>> @Override public Collection<InetSocketAddress> getRegisteredAddresses() throws IgniteSpiException {
>>>>>>> initClients();
>>>>>>>
>>>>>>> logger.debug("elb host discovery");
>>>>>>> List<String> privateIPs = awsService.getHealthyPrivateIPsFromTarget(targetGroupArn);
>>>>>>> List<InetSocketAddress> addrs = new ArrayList<>();
>>>>>>>
>>>>>>> for (String privateIP : privateIPs) {
>>>>>>> logger.debug("tcp discovery. found host "+privateIP);
>>>>>>> addrs.add(new InetSocketAddress(privateIP, 0));
>>>>>>> }
>>>>>>>
>>>>>>> return addrs;
>>>>>>> }
>>>>>>>
>>>>>>> @Override public void registerAddresses(Collection<InetSocketAddress> addrs) throws IgniteSpiException {
>>>>>>> //No-op, ELB will take care of registration.
>>>>>>> }
>>>>>>>
>>>>>>> @Override public void unregisterAddresses(Collection<InetSocketAddress> addrs) throws IgniteSpiException {
>>>>>>> // No-op, ELB will take care of this process.
>>>>>>> }
>>>>>>>
>>>>>>> And here is the ignite config setting
>>>>>>>
>>>>>>> TcpDiscoverySpi spi = new TcpDiscoverySpi();
>>>>>>>
>>>>>>> spi.setIpFinder(getTcpELBIpFinder(elbDiscoveryConfig));
>>>>>>>
>>>>>>> IgniteConfiguration igniteConfig = new IgniteConfiguration();
>>>>>>> igniteConfig.setMetricsLogFrequency(0);
>>>>>>> igniteConfig.setDiscoverySpi(spi);
>>>>>>> DataStorageConfiguration storageCfg = new DataStorageConfiguration();
>>>>>>> storageCfg.getDefaultDataRegionConfiguration().setPersistenceEnabled(false);
>>>>>>> igniteConfig.setDataStorageConfiguration(storageCfg);
>>>>>>>
>>>>>>> Any ideas?
>>>>>>>
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>>
>>>>>>> Ken
>>>>>>>
>>>>>>>
Re: Node joining didn't trigger Affinity recalculation problem
Posted by relax ken <re...@gmail.com>.
Hi Denis,
I just found an issue with the ec2 tcp discovery. If I have one node
running in elb and launch a new node in elb, they will join the same
cluster properly. However, if I launch two nodes separately first without
joining elb and then add them into the same elb after they are fully
running, they will find each other by ip finder but won't join the cluster.
Any ideas?
Thanks
On Sat, Apr 20, 2019 at 7:01 AM Denis Magda <dm...@apache.org> wrote:
> No problem, thanks for taking the time and sharing the issue. We'll update
> the docs. Good luck with your Ignite deployment!
>
> -
> Denis
>
>
> On Fri, Apr 19, 2019 at 10:32 PM relax ken <re...@gmail.com> wrote:
>
>> Great! Thanks Denis! that's the issue. After I configured it in security
>> group, it works
>>
>> On Sat, Apr 20, 2019 at 6:03 AM Denis Magda <dm...@apache.org> wrote:
>>
>>> By default, each node will be connecting to port 47500 and the next 100
>>> ports if nobody responds on 45700, 457001, etc.
>>>
>>> Please check that the port is opened. Alternatively, you can always
>>> manage the port with TcpDiscoverySpi.setLocalPort [1]. The ports range can
>>> be adjusted as well. [2]
>>>
>>> [1]
>>> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/TcpDiscoverySpi.java#L771
>>> [2]
>>> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/TcpDiscoverySpi.java#L782
>>>
>>> -
>>> Denis
>>>
>>>
>>> On Fri, Apr 19, 2019 at 9:38 PM relax ken <re...@gmail.com> wrote:
>>>
>>>> I don't think both nodes joined the cluster. That is the problem. I
>>>> don't know what it's wrong and why they are not in the cluster. They are in
>>>> the same subnet,. They can talk to each other. They found each other's IP.
>>>>
>>>> What port does ignite use for cluster joining? It may be security group
>>>> issue. Will ignite throw exception or log warning if they can't join the
>>>> cluster?
>>>>
>>>> Thanks
>>>>
>>>> On Sat, 20 Apr 2019, 05:50 Denis Magda, <dm...@apache.org> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Could you confirm that both nodes on AWS joined the cluster? Data will
>>>>> be rebalanced between the nodes once you see 2 server nodes in your cluster
>>>>> topology. Might be some networking issue.
>>>>>
>>>>> -
>>>>> Denis
>>>>>
>>>>>
>>>>> On Tue, Apr 16, 2019 at 9:53 AM relax ken <re...@gmail.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I wrote an aws ELB based tcp discovery ip finder. When I launched two
>>>>>> ec2 nodes in the same target group, the ip finder found node IPs correctly.
>>>>>> However, affinity recalculation wasn't triggered. Basically nothing
>>>>>> happened. also no exception. If I ran two nodes locally with multicast,
>>>>>> affinity recalculation worked properly. I just wonder what else I need
>>>>>> to do to make the custom IP finder works?
>>>>>>
>>>>>> Here is the overrided method in IPFinder. Basically it just is a
>>>>>> slightly modified version of TcpDiscoveryElbIpFinder
>>>>>>
>>>>>> @Override public Collection<InetSocketAddress> getRegisteredAddresses() throws IgniteSpiException {
>>>>>> initClients();
>>>>>>
>>>>>> logger.debug("elb host discovery");
>>>>>> List<String> privateIPs = awsService.getHealthyPrivateIPsFromTarget(targetGroupArn);
>>>>>> List<InetSocketAddress> addrs = new ArrayList<>();
>>>>>>
>>>>>> for (String privateIP : privateIPs) {
>>>>>> logger.debug("tcp discovery. found host "+privateIP);
>>>>>> addrs.add(new InetSocketAddress(privateIP, 0));
>>>>>> }
>>>>>>
>>>>>> return addrs;
>>>>>> }
>>>>>>
>>>>>> @Override public void registerAddresses(Collection<InetSocketAddress> addrs) throws IgniteSpiException {
>>>>>> //No-op, ELB will take care of registration.
>>>>>> }
>>>>>>
>>>>>> @Override public void unregisterAddresses(Collection<InetSocketAddress> addrs) throws IgniteSpiException {
>>>>>> // No-op, ELB will take care of this process.
>>>>>> }
>>>>>>
>>>>>> And here is the ignite config setting
>>>>>>
>>>>>> TcpDiscoverySpi spi = new TcpDiscoverySpi();
>>>>>>
>>>>>> spi.setIpFinder(getTcpELBIpFinder(elbDiscoveryConfig));
>>>>>>
>>>>>> IgniteConfiguration igniteConfig = new IgniteConfiguration();
>>>>>> igniteConfig.setMetricsLogFrequency(0);
>>>>>> igniteConfig.setDiscoverySpi(spi);
>>>>>> DataStorageConfiguration storageCfg = new DataStorageConfiguration();
>>>>>> storageCfg.getDefaultDataRegionConfiguration().setPersistenceEnabled(false);
>>>>>> igniteConfig.setDataStorageConfiguration(storageCfg);
>>>>>>
>>>>>> Any ideas?
>>>>>>
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>>
>>>>>> Ken
>>>>>>
>>>>>>
Re: Node joining didn't trigger Affinity recalculation problem
Posted by Denis Magda <dm...@apache.org>.
No problem, thanks for taking the time and sharing the issue. We'll update
the docs. Good luck with your Ignite deployment!
-
Denis
On Fri, Apr 19, 2019 at 10:32 PM relax ken <re...@gmail.com> wrote:
> Great! Thanks Denis! that's the issue. After I configured it in security
> group, it works
>
> On Sat, Apr 20, 2019 at 6:03 AM Denis Magda <dm...@apache.org> wrote:
>
>> By default, each node will be connecting to port 47500 and the next 100
>> ports if nobody responds on 45700, 457001, etc.
>>
>> Please check that the port is opened. Alternatively, you can always
>> manage the port with TcpDiscoverySpi.setLocalPort [1]. The ports range can
>> be adjusted as well. [2]
>>
>> [1]
>> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/TcpDiscoverySpi.java#L771
>> [2]
>> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/TcpDiscoverySpi.java#L782
>>
>> -
>> Denis
>>
>>
>> On Fri, Apr 19, 2019 at 9:38 PM relax ken <re...@gmail.com> wrote:
>>
>>> I don't think both nodes joined the cluster. That is the problem. I
>>> don't know what it's wrong and why they are not in the cluster. They are in
>>> the same subnet,. They can talk to each other. They found each other's IP.
>>>
>>> What port does ignite use for cluster joining? It may be security group
>>> issue. Will ignite throw exception or log warning if they can't join the
>>> cluster?
>>>
>>> Thanks
>>>
>>> On Sat, 20 Apr 2019, 05:50 Denis Magda, <dm...@apache.org> wrote:
>>>
>>>> Hi,
>>>>
>>>> Could you confirm that both nodes on AWS joined the cluster? Data will
>>>> be rebalanced between the nodes once you see 2 server nodes in your cluster
>>>> topology. Might be some networking issue.
>>>>
>>>> -
>>>> Denis
>>>>
>>>>
>>>> On Tue, Apr 16, 2019 at 9:53 AM relax ken <re...@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I wrote an aws ELB based tcp discovery ip finder. When I launched two
>>>>> ec2 nodes in the same target group, the ip finder found node IPs correctly.
>>>>> However, affinity recalculation wasn't triggered. Basically nothing
>>>>> happened. also no exception. If I ran two nodes locally with multicast,
>>>>> affinity recalculation worked properly. I just wonder what else I need
>>>>> to do to make the custom IP finder works?
>>>>>
>>>>> Here is the overrided method in IPFinder. Basically it just is a
>>>>> slightly modified version of TcpDiscoveryElbIpFinder
>>>>>
>>>>> @Override public Collection<InetSocketAddress> getRegisteredAddresses() throws IgniteSpiException {
>>>>> initClients();
>>>>>
>>>>> logger.debug("elb host discovery");
>>>>> List<String> privateIPs = awsService.getHealthyPrivateIPsFromTarget(targetGroupArn);
>>>>> List<InetSocketAddress> addrs = new ArrayList<>();
>>>>>
>>>>> for (String privateIP : privateIPs) {
>>>>> logger.debug("tcp discovery. found host "+privateIP);
>>>>> addrs.add(new InetSocketAddress(privateIP, 0));
>>>>> }
>>>>>
>>>>> return addrs;
>>>>> }
>>>>>
>>>>> @Override public void registerAddresses(Collection<InetSocketAddress> addrs) throws IgniteSpiException {
>>>>> //No-op, ELB will take care of registration.
>>>>> }
>>>>>
>>>>> @Override public void unregisterAddresses(Collection<InetSocketAddress> addrs) throws IgniteSpiException {
>>>>> // No-op, ELB will take care of this process.
>>>>> }
>>>>>
>>>>> And here is the ignite config setting
>>>>>
>>>>> TcpDiscoverySpi spi = new TcpDiscoverySpi();
>>>>>
>>>>> spi.setIpFinder(getTcpELBIpFinder(elbDiscoveryConfig));
>>>>>
>>>>> IgniteConfiguration igniteConfig = new IgniteConfiguration();
>>>>> igniteConfig.setMetricsLogFrequency(0);
>>>>> igniteConfig.setDiscoverySpi(spi);
>>>>> DataStorageConfiguration storageCfg = new DataStorageConfiguration();
>>>>> storageCfg.getDefaultDataRegionConfiguration().setPersistenceEnabled(false);
>>>>> igniteConfig.setDataStorageConfiguration(storageCfg);
>>>>>
>>>>> Any ideas?
>>>>>
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>> Ken
>>>>>
>>>>>
Re: Node joining didn't trigger Affinity recalculation problem
Posted by relax ken <re...@gmail.com>.
Great! Thanks Denis! that's the issue. After I configured it in security
group, it works
On Sat, Apr 20, 2019 at 6:03 AM Denis Magda <dm...@apache.org> wrote:
> By default, each node will be connecting to port 47500 and the next 100
> ports if nobody responds on 45700, 457001, etc.
>
> Please check that the port is opened. Alternatively, you can always manage
> the port with TcpDiscoverySpi.setLocalPort [1]. The ports range can be
> adjusted as well. [2]
>
> [1]
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/TcpDiscoverySpi.java#L771
> [2]
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/TcpDiscoverySpi.java#L782
>
> -
> Denis
>
>
> On Fri, Apr 19, 2019 at 9:38 PM relax ken <re...@gmail.com> wrote:
>
>> I don't think both nodes joined the cluster. That is the problem. I don't
>> know what it's wrong and why they are not in the cluster. They are in the
>> same subnet,. They can talk to each other. They found each other's IP.
>>
>> What port does ignite use for cluster joining? It may be security group
>> issue. Will ignite throw exception or log warning if they can't join the
>> cluster?
>>
>> Thanks
>>
>> On Sat, 20 Apr 2019, 05:50 Denis Magda, <dm...@apache.org> wrote:
>>
>>> Hi,
>>>
>>> Could you confirm that both nodes on AWS joined the cluster? Data will
>>> be rebalanced between the nodes once you see 2 server nodes in your cluster
>>> topology. Might be some networking issue.
>>>
>>> -
>>> Denis
>>>
>>>
>>> On Tue, Apr 16, 2019 at 9:53 AM relax ken <re...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I wrote an aws ELB based tcp discovery ip finder. When I launched two
>>>> ec2 nodes in the same target group, the ip finder found node IPs correctly.
>>>> However, affinity recalculation wasn't triggered. Basically nothing
>>>> happened. also no exception. If I ran two nodes locally with multicast,
>>>> affinity recalculation worked properly. I just wonder what else I need
>>>> to do to make the custom IP finder works?
>>>>
>>>> Here is the overrided method in IPFinder. Basically it just is a
>>>> slightly modified version of TcpDiscoveryElbIpFinder
>>>>
>>>> @Override public Collection<InetSocketAddress> getRegisteredAddresses() throws IgniteSpiException {
>>>> initClients();
>>>>
>>>> logger.debug("elb host discovery");
>>>> List<String> privateIPs = awsService.getHealthyPrivateIPsFromTarget(targetGroupArn);
>>>> List<InetSocketAddress> addrs = new ArrayList<>();
>>>>
>>>> for (String privateIP : privateIPs) {
>>>> logger.debug("tcp discovery. found host "+privateIP);
>>>> addrs.add(new InetSocketAddress(privateIP, 0));
>>>> }
>>>>
>>>> return addrs;
>>>> }
>>>>
>>>> @Override public void registerAddresses(Collection<InetSocketAddress> addrs) throws IgniteSpiException {
>>>> //No-op, ELB will take care of registration.
>>>> }
>>>>
>>>> @Override public void unregisterAddresses(Collection<InetSocketAddress> addrs) throws IgniteSpiException {
>>>> // No-op, ELB will take care of this process.
>>>> }
>>>>
>>>> And here is the ignite config setting
>>>>
>>>> TcpDiscoverySpi spi = new TcpDiscoverySpi();
>>>>
>>>> spi.setIpFinder(getTcpELBIpFinder(elbDiscoveryConfig));
>>>>
>>>> IgniteConfiguration igniteConfig = new IgniteConfiguration();
>>>> igniteConfig.setMetricsLogFrequency(0);
>>>> igniteConfig.setDiscoverySpi(spi);
>>>> DataStorageConfiguration storageCfg = new DataStorageConfiguration();
>>>> storageCfg.getDefaultDataRegionConfiguration().setPersistenceEnabled(false);
>>>> igniteConfig.setDataStorageConfiguration(storageCfg);
>>>>
>>>> Any ideas?
>>>>
>>>>
>>>> Thanks
>>>>
>>>>
>>>> Ken
>>>>
>>>>
Re: Node joining didn't trigger Affinity recalculation problem
Posted by Denis Magda <dm...@apache.org>.
By default, each node will be connecting to port 47500 and the next 100
ports if nobody responds on 45700, 457001, etc.
Please check that the port is opened. Alternatively, you can always manage
the port with TcpDiscoverySpi.setLocalPort [1]. The ports range can be
adjusted as well. [2]
[1]
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/TcpDiscoverySpi.java#L771
[2]
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/TcpDiscoverySpi.java#L782
-
Denis
On Fri, Apr 19, 2019 at 9:38 PM relax ken <re...@gmail.com> wrote:
> I don't think both nodes joined the cluster. That is the problem. I don't
> know what it's wrong and why they are not in the cluster. They are in the
> same subnet,. They can talk to each other. They found each other's IP.
>
> What port does ignite use for cluster joining? It may be security group
> issue. Will ignite throw exception or log warning if they can't join the
> cluster?
>
> Thanks
>
> On Sat, 20 Apr 2019, 05:50 Denis Magda, <dm...@apache.org> wrote:
>
>> Hi,
>>
>> Could you confirm that both nodes on AWS joined the cluster? Data will be
>> rebalanced between the nodes once you see 2 server nodes in your cluster
>> topology. Might be some networking issue.
>>
>> -
>> Denis
>>
>>
>> On Tue, Apr 16, 2019 at 9:53 AM relax ken <re...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I wrote an aws ELB based tcp discovery ip finder. When I launched two
>>> ec2 nodes in the same target group, the ip finder found node IPs correctly.
>>> However, affinity recalculation wasn't triggered. Basically nothing
>>> happened. also no exception. If I ran two nodes locally with multicast,
>>> affinity recalculation worked properly. I just wonder what else I need
>>> to do to make the custom IP finder works?
>>>
>>> Here is the overrided method in IPFinder. Basically it just is a
>>> slightly modified version of TcpDiscoveryElbIpFinder
>>>
>>> @Override public Collection<InetSocketAddress> getRegisteredAddresses() throws IgniteSpiException {
>>> initClients();
>>>
>>> logger.debug("elb host discovery");
>>> List<String> privateIPs = awsService.getHealthyPrivateIPsFromTarget(targetGroupArn);
>>> List<InetSocketAddress> addrs = new ArrayList<>();
>>>
>>> for (String privateIP : privateIPs) {
>>> logger.debug("tcp discovery. found host "+privateIP);
>>> addrs.add(new InetSocketAddress(privateIP, 0));
>>> }
>>>
>>> return addrs;
>>> }
>>>
>>> @Override public void registerAddresses(Collection<InetSocketAddress> addrs) throws IgniteSpiException {
>>> //No-op, ELB will take care of registration.
>>> }
>>>
>>> @Override public void unregisterAddresses(Collection<InetSocketAddress> addrs) throws IgniteSpiException {
>>> // No-op, ELB will take care of this process.
>>> }
>>>
>>> And here is the ignite config setting
>>>
>>> TcpDiscoverySpi spi = new TcpDiscoverySpi();
>>>
>>> spi.setIpFinder(getTcpELBIpFinder(elbDiscoveryConfig));
>>>
>>> IgniteConfiguration igniteConfig = new IgniteConfiguration();
>>> igniteConfig.setMetricsLogFrequency(0);
>>> igniteConfig.setDiscoverySpi(spi);
>>> DataStorageConfiguration storageCfg = new DataStorageConfiguration();
>>> storageCfg.getDefaultDataRegionConfiguration().setPersistenceEnabled(false);
>>> igniteConfig.setDataStorageConfiguration(storageCfg);
>>>
>>> Any ideas?
>>>
>>>
>>> Thanks
>>>
>>>
>>> Ken
>>>
>>>
Re: Node joining didn't trigger Affinity recalculation problem
Posted by relax ken <re...@gmail.com>.
I don't think both nodes joined the cluster. That is the problem. I don't
know what it's wrong and why they are not in the cluster. They are in the
same subnet,. They can talk to each other. They found each other's IP.
What port does ignite use for cluster joining? It may be security group
issue. Will ignite throw exception or log warning if they can't join the
cluster?
Thanks
On Sat, 20 Apr 2019, 05:50 Denis Magda, <dm...@apache.org> wrote:
> Hi,
>
> Could you confirm that both nodes on AWS joined the cluster? Data will be
> rebalanced between the nodes once you see 2 server nodes in your cluster
> topology. Might be some networking issue.
>
> -
> Denis
>
>
> On Tue, Apr 16, 2019 at 9:53 AM relax ken <re...@gmail.com> wrote:
>
>> Hi,
>>
>> I wrote an aws ELB based tcp discovery ip finder. When I launched two ec2
>> nodes in the same target group, the ip finder found node IPs correctly.
>> However, affinity recalculation wasn't triggered. Basically nothing
>> happened. also no exception. If I ran two nodes locally with multicast,
>> affinity recalculation worked properly. I just wonder what else I need
>> to do to make the custom IP finder works?
>>
>> Here is the overrided method in IPFinder. Basically it just is a slightly
>> modified version of TcpDiscoveryElbIpFinder
>>
>> @Override public Collection<InetSocketAddress> getRegisteredAddresses() throws IgniteSpiException {
>> initClients();
>>
>> logger.debug("elb host discovery");
>> List<String> privateIPs = awsService.getHealthyPrivateIPsFromTarget(targetGroupArn);
>> List<InetSocketAddress> addrs = new ArrayList<>();
>>
>> for (String privateIP : privateIPs) {
>> logger.debug("tcp discovery. found host "+privateIP);
>> addrs.add(new InetSocketAddress(privateIP, 0));
>> }
>>
>> return addrs;
>> }
>>
>> @Override public void registerAddresses(Collection<InetSocketAddress> addrs) throws IgniteSpiException {
>> //No-op, ELB will take care of registration.
>> }
>>
>> @Override public void unregisterAddresses(Collection<InetSocketAddress> addrs) throws IgniteSpiException {
>> // No-op, ELB will take care of this process.
>> }
>>
>> And here is the ignite config setting
>>
>> TcpDiscoverySpi spi = new TcpDiscoverySpi();
>>
>> spi.setIpFinder(getTcpELBIpFinder(elbDiscoveryConfig));
>>
>> IgniteConfiguration igniteConfig = new IgniteConfiguration();
>> igniteConfig.setMetricsLogFrequency(0);
>> igniteConfig.setDiscoverySpi(spi);
>> DataStorageConfiguration storageCfg = new DataStorageConfiguration();
>> storageCfg.getDefaultDataRegionConfiguration().setPersistenceEnabled(false);
>> igniteConfig.setDataStorageConfiguration(storageCfg);
>>
>> Any ideas?
>>
>>
>> Thanks
>>
>>
>> Ken
>>
>>
Re: Node joining didn't trigger Affinity recalculation problem
Posted by Denis Magda <dm...@apache.org>.
Hi,
Could you confirm that both nodes on AWS joined the cluster? Data will be
rebalanced between the nodes once you see 2 server nodes in your cluster
topology. Might be some networking issue.
-
Denis
On Tue, Apr 16, 2019 at 9:53 AM relax ken <re...@gmail.com> wrote:
> Hi,
>
> I wrote an aws ELB based tcp discovery ip finder. When I launched two ec2
> nodes in the same target group, the ip finder found node IPs correctly.
> However, affinity recalculation wasn't triggered. Basically nothing
> happened. also no exception. If I ran two nodes locally with multicast,
> affinity recalculation worked properly. I just wonder what else I need
> to do to make the custom IP finder works?
>
> Here is the overrided method in IPFinder. Basically it just is a slightly
> modified version of TcpDiscoveryElbIpFinder
>
> @Override public Collection<InetSocketAddress> getRegisteredAddresses() throws IgniteSpiException {
> initClients();
>
> logger.debug("elb host discovery");
> List<String> privateIPs = awsService.getHealthyPrivateIPsFromTarget(targetGroupArn);
> List<InetSocketAddress> addrs = new ArrayList<>();
>
> for (String privateIP : privateIPs) {
> logger.debug("tcp discovery. found host "+privateIP);
> addrs.add(new InetSocketAddress(privateIP, 0));
> }
>
> return addrs;
> }
>
> @Override public void registerAddresses(Collection<InetSocketAddress> addrs) throws IgniteSpiException {
> //No-op, ELB will take care of registration.
> }
>
> @Override public void unregisterAddresses(Collection<InetSocketAddress> addrs) throws IgniteSpiException {
> // No-op, ELB will take care of this process.
> }
>
> And here is the ignite config setting
>
> TcpDiscoverySpi spi = new TcpDiscoverySpi();
>
> spi.setIpFinder(getTcpELBIpFinder(elbDiscoveryConfig));
>
> IgniteConfiguration igniteConfig = new IgniteConfiguration();
> igniteConfig.setMetricsLogFrequency(0);
> igniteConfig.setDiscoverySpi(spi);
> DataStorageConfiguration storageCfg = new DataStorageConfiguration();
> storageCfg.getDefaultDataRegionConfiguration().setPersistenceEnabled(false);
> igniteConfig.setDataStorageConfiguration(storageCfg);
>
> Any ideas?
>
>
> Thanks
>
>
> Ken
>
>