You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Lukas Polacek <lu...@innovatrics.com> on 2019/01/03 10:17:55 UTC
Iterating through native persistence entries before joining the cluster
Hi,
in our use case we need to run some C++ code (via JNI) whenever something
is pushed into the local Ignite cache. In other words, we need to have
Ignite in sync with C++ memory. We have a local listener that listens to
EVT_CACHE_OBJECT_PUT events and executes the C++ code, so everything is
fine while the node is running. However, we use native persistence, so
after a node restart, the local cache is read from the disk but the C++
code hasn't been run for any cache entries, which means that Ignite and C++
memory are out of sync.
Iterating through the local cache entries is only possible once the node
has already joined the cluster, but that's too late for us - it needs to be
done before joining the cluster.
I've managed to add a lifecycle bean event BEFORE_CLUSTER_JOIN (see
http://apache-ignite-users.70518.x6.nabble.com/Register-listeners-before-joining-the-cluster-tc25944.html,
a PR is hopefully coming soon), which is triggered before joining the
cluster but at that point we cannot access the cache via ignite.cache(...).
Is there a way to access all entries in the native persistence at that
point or earlier? I'm also fine with modifying the Ignite source code if
that's necessary (and simple enough), since we are just prototyping.
Re: Iterating through native persistence entries before joining the cluster
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I guess you could be starting a service instance on every node, doing
warmup in init().
Then service requests will be only routed on nodes where warmup is complete.
Regards,
--
Ilya Kasnacheev
чт, 17 янв. 2019 г. в 13:26, Lukas Polacek <lu...@innovatrics.com>:
> Technically, I could delay joining the cluster by adding Thread.sleep()
> into the BEFORE_NODE_START lifecycle bean (or my newly
> created BEFORE_CLUSTER_JOIN). I'd like to iterate through the local cache
> in one of those lifecycle beans - but that doesn't seem possible.
>
> Regarding cluster groups: I can't use IgniteCluster::forAttributes, since
> attributes are set at the node start. The only remaining option seems
> IgniteCluster::forPredicate, but that's also not very helpful, since I
> don't see how to leverage ClusterNode's properties, in particular its
> metrics.
>
> We have been using IgniteSet as mentioned above to produce an "active
> cluster group" but I'm looking for something simpler. Today I found a very
> problematic case which will require yet another special logic to handle it.
>
> On Fri, Jan 11, 2019 at 4:14 PM Ilya Kasnacheev <il...@gmail.com>
> wrote:
>
>> Hello!
>>
>> Actually, delaying joining the cluster is complex enough since it will
>> add a lot of unexpected concerns.
>>
>> There are ClusterGroup's precisely for that purpose.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пт, 11 янв. 2019 г. в 15:58, Lukas Polacek <lukas.polacek@innovatrics.com
>> >:
>>
>>> Hi,
>>> we already do that by using an IgniteSet which contains consistent IDs
>>> of the nodes that are ready, but it's very error-prone and unnecessarily
>>> complex. Delaying joining the cluster would make everything simpler.
>>>
>>> On Thu, Jan 10, 2019 at 1:38 PM Ilya Kasnacheev <
>>> ilya.kasnacheev@gmail.com> wrote:
>>>
>>>> Hello!
>>>>
>>>> I think you should decouple workload readiness from joining the cluster.
>>>>
>>>> You should let node join cluster first, then iterate entries, and only
>>>> then allow workload to this node.
>>>>
>>>> Regards,
>>>> --
>>>> Ilya Kasnacheev
>>>>
>>>>
>>>> чт, 3 янв. 2019 г. в 13:18, Lukas Polacek <
>>>> lukas.polacek@innovatrics.com>:
>>>>
>>>>> Hi,
>>>>> in our use case we need to run some C++ code (via JNI) whenever
>>>>> something is pushed into the local Ignite cache. In other words, we need to
>>>>> have Ignite in sync with C++ memory. We have a local listener that listens
>>>>> to EVT_CACHE_OBJECT_PUT events and executes the C++ code, so everything is
>>>>> fine while the node is running. However, we use native persistence, so
>>>>> after a node restart, the local cache is read from the disk but the C++
>>>>> code hasn't been run for any cache entries, which means that Ignite and C++
>>>>> memory are out of sync.
>>>>>
>>>>> Iterating through the local cache entries is only possible once the
>>>>> node has already joined the cluster, but that's too late for us - it needs
>>>>> to be done before joining the cluster.
>>>>>
>>>>> I've managed to add a lifecycle bean event BEFORE_CLUSTER_JOIN (see
>>>>> http://apache-ignite-users.70518.x6.nabble.com/Register-listeners-before-joining-the-cluster-tc25944.html,
>>>>> a PR is hopefully coming soon), which is triggered before joining the
>>>>> cluster but at that point we cannot access the cache via
>>>>> ignite.cache(...). Is there a way to access all entries in the native
>>>>> persistence at that point or earlier? I'm also fine with modifying the
>>>>> Ignite source code if that's necessary (and simple enough), since we are
>>>>> just prototyping.
>>>>>
>>>>
Re: Iterating through native persistence entries before joining the cluster
Posted by Lukas Polacek <lu...@innovatrics.com>.
Technically, I could delay joining the cluster by adding Thread.sleep()
into the BEFORE_NODE_START lifecycle bean (or my newly
created BEFORE_CLUSTER_JOIN). I'd like to iterate through the local cache
in one of those lifecycle beans - but that doesn't seem possible.
Regarding cluster groups: I can't use IgniteCluster::forAttributes, since
attributes are set at the node start. The only remaining option seems
IgniteCluster::forPredicate, but that's also not very helpful, since I
don't see how to leverage ClusterNode's properties, in particular its
metrics.
We have been using IgniteSet as mentioned above to produce an "active
cluster group" but I'm looking for something simpler. Today I found a very
problematic case which will require yet another special logic to handle it.
On Fri, Jan 11, 2019 at 4:14 PM Ilya Kasnacheev <il...@gmail.com>
wrote:
> Hello!
>
> Actually, delaying joining the cluster is complex enough since it will add
> a lot of unexpected concerns.
>
> There are ClusterGroup's precisely for that purpose.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 11 янв. 2019 г. в 15:58, Lukas Polacek <lukas.polacek@innovatrics.com
> >:
>
>> Hi,
>> we already do that by using an IgniteSet which contains consistent IDs of
>> the nodes that are ready, but it's very error-prone and unnecessarily
>> complex. Delaying joining the cluster would make everything simpler.
>>
>> On Thu, Jan 10, 2019 at 1:38 PM Ilya Kasnacheev <
>> ilya.kasnacheev@gmail.com> wrote:
>>
>>> Hello!
>>>
>>> I think you should decouple workload readiness from joining the cluster.
>>>
>>> You should let node join cluster first, then iterate entries, and only
>>> then allow workload to this node.
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> чт, 3 янв. 2019 г. в 13:18, Lukas Polacek <lukas.polacek@innovatrics.com
>>> >:
>>>
>>>> Hi,
>>>> in our use case we need to run some C++ code (via JNI) whenever
>>>> something is pushed into the local Ignite cache. In other words, we need to
>>>> have Ignite in sync with C++ memory. We have a local listener that listens
>>>> to EVT_CACHE_OBJECT_PUT events and executes the C++ code, so everything is
>>>> fine while the node is running. However, we use native persistence, so
>>>> after a node restart, the local cache is read from the disk but the C++
>>>> code hasn't been run for any cache entries, which means that Ignite and C++
>>>> memory are out of sync.
>>>>
>>>> Iterating through the local cache entries is only possible once the
>>>> node has already joined the cluster, but that's too late for us - it needs
>>>> to be done before joining the cluster.
>>>>
>>>> I've managed to add a lifecycle bean event BEFORE_CLUSTER_JOIN (see
>>>> http://apache-ignite-users.70518.x6.nabble.com/Register-listeners-before-joining-the-cluster-tc25944.html,
>>>> a PR is hopefully coming soon), which is triggered before joining the
>>>> cluster but at that point we cannot access the cache via
>>>> ignite.cache(...). Is there a way to access all entries in the native
>>>> persistence at that point or earlier? I'm also fine with modifying the
>>>> Ignite source code if that's necessary (and simple enough), since we are
>>>> just prototyping.
>>>>
>>>
Re: Iterating through native persistence entries before joining the cluster
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Actually, delaying joining the cluster is complex enough since it will add
a lot of unexpected concerns.
There are ClusterGroup's precisely for that purpose.
Regards,
--
Ilya Kasnacheev
пт, 11 янв. 2019 г. в 15:58, Lukas Polacek <lu...@innovatrics.com>:
> Hi,
> we already do that by using an IgniteSet which contains consistent IDs of
> the nodes that are ready, but it's very error-prone and unnecessarily
> complex. Delaying joining the cluster would make everything simpler.
>
> On Thu, Jan 10, 2019 at 1:38 PM Ilya Kasnacheev <il...@gmail.com>
> wrote:
>
>> Hello!
>>
>> I think you should decouple workload readiness from joining the cluster.
>>
>> You should let node join cluster first, then iterate entries, and only
>> then allow workload to this node.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> чт, 3 янв. 2019 г. в 13:18, Lukas Polacek <lukas.polacek@innovatrics.com
>> >:
>>
>>> Hi,
>>> in our use case we need to run some C++ code (via JNI) whenever
>>> something is pushed into the local Ignite cache. In other words, we need to
>>> have Ignite in sync with C++ memory. We have a local listener that listens
>>> to EVT_CACHE_OBJECT_PUT events and executes the C++ code, so everything is
>>> fine while the node is running. However, we use native persistence, so
>>> after a node restart, the local cache is read from the disk but the C++
>>> code hasn't been run for any cache entries, which means that Ignite and C++
>>> memory are out of sync.
>>>
>>> Iterating through the local cache entries is only possible once the node
>>> has already joined the cluster, but that's too late for us - it needs to be
>>> done before joining the cluster.
>>>
>>> I've managed to add a lifecycle bean event BEFORE_CLUSTER_JOIN (see
>>> http://apache-ignite-users.70518.x6.nabble.com/Register-listeners-before-joining-the-cluster-tc25944.html,
>>> a PR is hopefully coming soon), which is triggered before joining the
>>> cluster but at that point we cannot access the cache via
>>> ignite.cache(...). Is there a way to access all entries in the native
>>> persistence at that point or earlier? I'm also fine with modifying the
>>> Ignite source code if that's necessary (and simple enough), since we are
>>> just prototyping.
>>>
>>
Re: Iterating through native persistence entries before joining the cluster
Posted by Lukas Polacek <lu...@innovatrics.com>.
Hi,
we already do that by using an IgniteSet which contains consistent IDs of
the nodes that are ready, but it's very error-prone and unnecessarily
complex. Delaying joining the cluster would make everything simpler.
On Thu, Jan 10, 2019 at 1:38 PM Ilya Kasnacheev <il...@gmail.com>
wrote:
> Hello!
>
> I think you should decouple workload readiness from joining the cluster.
>
> You should let node join cluster first, then iterate entries, and only
> then allow workload to this node.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> чт, 3 янв. 2019 г. в 13:18, Lukas Polacek <lu...@innovatrics.com>:
>
>> Hi,
>> in our use case we need to run some C++ code (via JNI) whenever something
>> is pushed into the local Ignite cache. In other words, we need to have
>> Ignite in sync with C++ memory. We have a local listener that listens to
>> EVT_CACHE_OBJECT_PUT events and executes the C++ code, so everything is
>> fine while the node is running. However, we use native persistence, so
>> after a node restart, the local cache is read from the disk but the C++
>> code hasn't been run for any cache entries, which means that Ignite and C++
>> memory are out of sync.
>>
>> Iterating through the local cache entries is only possible once the node
>> has already joined the cluster, but that's too late for us - it needs to be
>> done before joining the cluster.
>>
>> I've managed to add a lifecycle bean event BEFORE_CLUSTER_JOIN (see
>> http://apache-ignite-users.70518.x6.nabble.com/Register-listeners-before-joining-the-cluster-tc25944.html,
>> a PR is hopefully coming soon), which is triggered before joining the
>> cluster but at that point we cannot access the cache via
>> ignite.cache(...). Is there a way to access all entries in the native
>> persistence at that point or earlier? I'm also fine with modifying the
>> Ignite source code if that's necessary (and simple enough), since we are
>> just prototyping.
>>
>
Re: Iterating through native persistence entries before joining the cluster
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I think you should decouple workload readiness from joining the cluster.
You should let node join cluster first, then iterate entries, and only then
allow workload to this node.
Regards,
--
Ilya Kasnacheev
чт, 3 янв. 2019 г. в 13:18, Lukas Polacek <lu...@innovatrics.com>:
> Hi,
> in our use case we need to run some C++ code (via JNI) whenever something
> is pushed into the local Ignite cache. In other words, we need to have
> Ignite in sync with C++ memory. We have a local listener that listens to
> EVT_CACHE_OBJECT_PUT events and executes the C++ code, so everything is
> fine while the node is running. However, we use native persistence, so
> after a node restart, the local cache is read from the disk but the C++
> code hasn't been run for any cache entries, which means that Ignite and C++
> memory are out of sync.
>
> Iterating through the local cache entries is only possible once the node
> has already joined the cluster, but that's too late for us - it needs to be
> done before joining the cluster.
>
> I've managed to add a lifecycle bean event BEFORE_CLUSTER_JOIN (see
> http://apache-ignite-users.70518.x6.nabble.com/Register-listeners-before-joining-the-cluster-tc25944.html,
> a PR is hopefully coming soon), which is triggered before joining the
> cluster but at that point we cannot access the cache via ignite.cache(...).
> Is there a way to access all entries in the native persistence at that
> point or earlier? I'm also fine with modifying the Ignite source code if
> that's necessary (and simple enough), since we are just prototyping.
>