You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Humphrey <hm...@gmail.com> on 2020/07/07 09:42:26 UTC
Force backup on different physical machine
Let say I have 2 kubernetes nodes and I have 4 ignite server nodes.
This will result (if kubernetes have 2 pods running on each node) in the
following:
*kubernetes_node1:* ignite_node1, ignite_node2
*kubernetes_node2:* ignite_node3, ignite_node4
I specify that my cache backup = 1
Is there a way to configure that the backup data of the ignite_node1 goes on
the ignite_node3 or ignite_node4 and NOT on ignite_node2 (same physical
machine/kubernetes node)? Is there any configuration for this (I assume it's
something on runtime, because we don't know where kubernetes will schedule
the pod)?
Background:
If kubernetes_node1 goes down, then there won't be any data loss.
Humphrey
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Force backup on different physical machine
Posted by Pavel Tupitsyn <pt...@apache.org>.
Normally there are two ways to achieve this: excludeNeighbors
and AffinityBackupFilter [1]
However, excludeNeighbors won't work across pods on the same k8s node,
since it relies on MAC addresses.
So your best bet is to use ClusterNodeAttributeAffinityBackupFilter:
* set ClusterNodeAttributeAffinityBackupFilter with env var K8S_NODE_NAME
as described in [2]
* export K8S_NODE_NAME environment variable in Kubernetes as described in
[3]:
env:
- name: K8S_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
This way backups won't end up on the same k8s node.
[1]
https://apacheignite.readme.io/docs/affinity-collocation#crash-safe-affinity
[2]
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/affinity/rendezvous/ClusterNodeAttributeAffinityBackupFilter.html
[3]
https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/#use-pod-fields-as-values-for-environment-variables
On Tue, Jul 7, 2020 at 12:42 PM Humphrey <hm...@gmail.com> wrote:
> Let say I have 2 kubernetes nodes and I have 4 ignite server nodes.
> This will result (if kubernetes have 2 pods running on each node) in the
> following:
>
> *kubernetes_node1:* ignite_node1, ignite_node2
> *kubernetes_node2:* ignite_node3, ignite_node4
>
> I specify that my cache backup = 1
>
> Is there a way to configure that the backup data of the ignite_node1 goes
> on
> the ignite_node3 or ignite_node4 and NOT on ignite_node2 (same physical
> machine/kubernetes node)? Is there any configuration for this (I assume
> it's
> something on runtime, because we don't know where kubernetes will schedule
> the pod)?
>
> Background:
> If kubernetes_node1 goes down, then there won't be any data loss.
>
> Humphrey
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Force backup on different physical machine
Posted by narges saleh <sn...@gmail.com>.
Thanks Stephan.
I am saying one ignite-node/pod/k8 node spread across availability zones
On Tue, Jul 21, 2020 at 9:51 AM Stephen Darlington <
stephen.darlington@gridgain.com> wrote:
> 1) I think you’re saying you have two Ignite nodes on one physical
> machine. What I mean is, don’t do that. All else being equal, one server
> node per physical machine is the way to go
>
> 2) This is an Ignite feature and not limited to any particular cloud
> vendor. It does rely on your cloud vendor “publishing” the underlying
> machine (or availability zone) as an environment variable so Ignite can
> find it
>
> On 21 Jul 2020, at 14:01, narges saleh <sn...@gmail.com> wrote:
>
> Hi Stephen,
> 1) [Having said that, you’re probably best configuring k8s not to put two
> Ignite server nodes on a single machine.]
> Do you mean by having k8s place the VMs in different availability zones?
>
> Then how would you tell ignite to place the backups in a different zone?
> I've read this is possible via affinityBackupFilter. Which brings up my
> next question.
>
> 2) Is affinityBackupFilter available with all public cloud platforms or
> just AWS?
>
> thanks.
>
> On Tue, Jul 7, 2020 at 5:20 AM Stephen Darlington <
> stephen.darlington@gridgain.com> wrote:
>
>> You can configure the affinity function (RendezvousAffinityFunction
>> <https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/affinity/rendezvous/RendezvousAffinityFunction.html>).
>> If you set the backup filter, you can customise which nodes are considered
>> for use as backups:
>>
>> cacheConfiguration.setBackups(1);
>> cacheConfiguration.setAffinity(new RendezvousAffinityFunction(1024, (n,p) -> {
>> return !p.hostNames().equals(n.hostNames());
>> }));
>>
>> Having said that, you’re probably best configuring k8s not to put two
>> Ignite server nodes on a single machine.
>>
>> Regards,
>> Stephen
>>
>> On 7 Jul 2020, at 10:42, Humphrey <hm...@gmail.com> wrote:
>>
>> Let say I have 2 kubernetes nodes and I have 4 ignite server nodes.
>> This will result (if kubernetes have 2 pods running on each node) in the
>> following:
>>
>> *kubernetes_node1:* ignite_node1, ignite_node2
>> *kubernetes_node2:* ignite_node3, ignite_node4
>>
>> I specify that my cache backup = 1
>>
>> Is there a way to configure that the backup data of the ignite_node1 goes
>> on
>> the ignite_node3 or ignite_node4 and NOT on ignite_node2 (same physical
>> machine/kubernetes node)? Is there any configuration for this (I assume
>> it's
>> something on runtime, because we don't know where kubernetes will schedule
>> the pod)?
>>
>> Background:
>> If kubernetes_node1 goes down, then there won't be any data loss.
>>
>> Humphrey
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>>
>>
>>
>
>
Re: Force backup on different physical machine
Posted by Stephen Darlington <st...@gridgain.com>.
1) I think you’re saying you have two Ignite nodes on one physical machine. What I mean is, don’t do that. All else being equal, one server node per physical machine is the way to go
2) This is an Ignite feature and not limited to any particular cloud vendor. It does rely on your cloud vendor “publishing” the underlying machine (or availability zone) as an environment variable so Ignite can find it
> On 21 Jul 2020, at 14:01, narges saleh <sn...@gmail.com> wrote:
>
> Hi Stephen,
> 1) [Having said that, you’re probably best configuring k8s not to put two Ignite server nodes on a single machine.]
> Do you mean by having k8s place the VMs in different availability zones?
>
> Then how would you tell ignite to place the backups in a different zone?
> I've read this is possible via affinityBackupFilter. Which brings up my next question.
>
> 2) Is affinityBackupFilter available with all public cloud platforms or just AWS?
>
> thanks.
>
> On Tue, Jul 7, 2020 at 5:20 AM Stephen Darlington <stephen.darlington@gridgain.com <ma...@gridgain.com>> wrote:
> You can configure the affinity function (RendezvousAffinityFunction <https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/affinity/rendezvous/RendezvousAffinityFunction.html>). If you set the backup filter, you can customise which nodes are considered for use as backups:
> cacheConfiguration.setBackups(1);
> cacheConfiguration.setAffinity(new RendezvousAffinityFunction(1024, (n,p) -> {
> return !p.hostNames().equals(n.hostNames());
> }));
> Having said that, you’re probably best configuring k8s not to put two Ignite server nodes on a single machine.
>
> Regards,
> Stephen
>
>> On 7 Jul 2020, at 10:42, Humphrey <hmmlopez@gmail.com <ma...@gmail.com>> wrote:
>>
>> Let say I have 2 kubernetes nodes and I have 4 ignite server nodes.
>> This will result (if kubernetes have 2 pods running on each node) in the
>> following:
>>
>> *kubernetes_node1:* ignite_node1, ignite_node2
>> *kubernetes_node2:* ignite_node3, ignite_node4
>>
>> I specify that my cache backup = 1
>>
>> Is there a way to configure that the backup data of the ignite_node1 goes on
>> the ignite_node3 or ignite_node4 and NOT on ignite_node2 (same physical
>> machine/kubernetes node)? Is there any configuration for this (I assume it's
>> something on runtime, because we don't know where kubernetes will schedule
>> the pod)?
>>
>> Background:
>> If kubernetes_node1 goes down, then there won't be any data loss.
>>
>> Humphrey
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ <http://apache-ignite-users.70518.x6.nabble.com/>
>
>
Re: Force backup on different physical machine
Posted by narges saleh <sn...@gmail.com>.
Hi Stephen,
1) [Having said that, you’re probably best configuring k8s not to put two
Ignite server nodes on a single machine.]
Do you mean by having k8s place the VMs in different availability zones?
Then how would you tell ignite to place the backups in a different zone?
I've read this is possible via affinityBackupFilter. Which brings up my
next question.
2) Is affinityBackupFilter available with all public cloud platforms or
just AWS?
thanks.
On Tue, Jul 7, 2020 at 5:20 AM Stephen Darlington <
stephen.darlington@gridgain.com> wrote:
> You can configure the affinity function (RendezvousAffinityFunction
> <https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/affinity/rendezvous/RendezvousAffinityFunction.html>).
> If you set the backup filter, you can customise which nodes are considered
> for use as backups:
>
> cacheConfiguration.setBackups(1);
> cacheConfiguration.setAffinity(new RendezvousAffinityFunction(1024, (n,p) -> {
> return !p.hostNames().equals(n.hostNames());
> }));
>
> Having said that, you’re probably best configuring k8s not to put two
> Ignite server nodes on a single machine.
>
> Regards,
> Stephen
>
> On 7 Jul 2020, at 10:42, Humphrey <hm...@gmail.com> wrote:
>
> Let say I have 2 kubernetes nodes and I have 4 ignite server nodes.
> This will result (if kubernetes have 2 pods running on each node) in the
> following:
>
> *kubernetes_node1:* ignite_node1, ignite_node2
> *kubernetes_node2:* ignite_node3, ignite_node4
>
> I specify that my cache backup = 1
>
> Is there a way to configure that the backup data of the ignite_node1 goes
> on
> the ignite_node3 or ignite_node4 and NOT on ignite_node2 (same physical
> machine/kubernetes node)? Is there any configuration for this (I assume
> it's
> something on runtime, because we don't know where kubernetes will schedule
> the pod)?
>
> Background:
> If kubernetes_node1 goes down, then there won't be any data loss.
>
> Humphrey
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
>
>
>
Re: Force backup on different physical machine
Posted by Stephen Darlington <st...@gridgain.com>.
You can configure the affinity function (RendezvousAffinityFunction <https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/affinity/rendezvous/RendezvousAffinityFunction.html>). If you set the backup filter, you can customise which nodes are considered for use as backups:
cacheConfiguration.setBackups(1);
cacheConfiguration.setAffinity(new RendezvousAffinityFunction(1024, (n,p) -> {
return !p.hostNames().equals(n.hostNames());
}));
Having said that, you’re probably best configuring k8s not to put two Ignite server nodes on a single machine.
Regards,
Stephen
> On 7 Jul 2020, at 10:42, Humphrey <hm...@gmail.com> wrote:
>
> Let say I have 2 kubernetes nodes and I have 4 ignite server nodes.
> This will result (if kubernetes have 2 pods running on each node) in the
> following:
>
> *kubernetes_node1:* ignite_node1, ignite_node2
> *kubernetes_node2:* ignite_node3, ignite_node4
>
> I specify that my cache backup = 1
>
> Is there a way to configure that the backup data of the ignite_node1 goes on
> the ignite_node3 or ignite_node4 and NOT on ignite_node2 (same physical
> machine/kubernetes node)? Is there any configuration for this (I assume it's
> something on runtime, because we don't know where kubernetes will schedule
> the pod)?
>
> Background:
> If kubernetes_node1 goes down, then there won't be any data loss.
>
> Humphrey
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/