You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by zbyszek <zb...@yahoo.com> on 2018/01/30 16:49:28 UTC
IndexingSpi#remove not called on key migration
I am testing IndexingSpi in cluster of 2 nodes for the the following cache
config:
private IgniteCache<String, BinaryObject> createCache() {
CacheConfiguration<String, BinaryObject> cCfg = new
CacheConfiguration<>();
cCfg.setName("MyCache");
cCfg.setStoreKeepBinary(true);
cCfg.setCacheMode(CacheMode.PARTITIONED);
cCfg.setOnheapCacheEnabled(false);
cCfg.setCopyOnRead(false);
cCfg.setBackups(1);
cCfg.setWriteBehindEnabled(false);
cCfg.setReadThrough(false);
return ignite.getOrCreateCache(cCfg).withKeepBinary();
}
I have observed that method org.apache.ignite.spi.indexing.IndexingSpi#store
is called for all keys, both primay and backup,
resulting in both nodes having built duplicated index. In addition,
org.apache.ignite.spi.indexing.IndexingSpi#remove is not called when the
keys migrate to another node (for example due to second node joining the
cluster).
Is that by design or rather a bug? I would expect the receive remove()
callback when primary keys migrate from the node, so I can free resources
occupied by corresponding index.
Thank you in advance for your help,
zbyszek
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: IndexingSpi#remove not called on key migration
Posted by Valentin Kulichenko <va...@gmail.com>.
Zbyszek,
I don't there is an issue. You start two nodes with one backup, therefore
eviction never happens. You basically have a fully replicated dataset at
this point. Try to start a third node, you should start getting remove()
callbacks then.
Indexes on backups are needed for failover. If primary node fails, you
still have the data in cache as well as indexes for this data.
-Val
On Fri, Feb 2, 2018 at 4:08 AM, zbyszek <zb...@yahoo.com> wrote:
> Hello Andrew,
> Thank you for your explanation.
>
> Where do you propose to use the affinity check - during query processing or
> in the store() method? If in store() - how this would solve the issue where
> first node starts long before second one and there is no callback on the
> first node to get informed about keys migration?
>
> In addition, what would be the use case that would require calling store()
> for backup keys?
>
>
> Thank you again,
> Zbyszek
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: IndexingSpi#remove not called on key migration
Posted by zbyszek <zb...@yahoo.com>.
Hello Andrew,
Thank you for your explanation.
Where do you propose to use the affinity check - during query processing or
in the store() method? If in store() - how this would solve the issue where
first node starts long before second one and there is no callback on the
first node to get informed about keys migration?
In addition, what would be the use case that would require calling store()
for backup keys?
Thank you again,
Zbyszek
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: IndexingSpi#remove not called on key migration
Posted by Andrey Mashenkov <an...@gmail.com>.
Hi,
You have 2 nodes and Partitioned index with 1 backup and both nodes should
have full dataset.
So, when a new node is added Ignite will copy data to the new node and no
partitions will be evicted.
IndexingSpi#remove() method is called only when entry removed from node
(either deleted from cache or due to partition eviction)
and it is ok IndexingSpi#store() is called for primary and backups keys.
You can inject Ignite instance into your IndexingSpi implementation
via @IgniteInstanceResource annotation
and use ignite.affinity() information to check if node is primary or backup
for key.
On Tue, Jan 30, 2018 at 7:49 PM, zbyszek <zb...@yahoo.com> wrote:
>
> I am testing IndexingSpi in cluster of 2 nodes for the the following cache
> config:
>
> private IgniteCache<String, BinaryObject> createCache() {
> CacheConfiguration<String, BinaryObject> cCfg = new
> CacheConfiguration<>();
> cCfg.setName("MyCache");
> cCfg.setStoreKeepBinary(true);
> cCfg.setCacheMode(CacheMode.PARTITIONED);
> cCfg.setOnheapCacheEnabled(false);
> cCfg.setCopyOnRead(false);
> cCfg.setBackups(1);
> cCfg.setWriteBehindEnabled(false);
> cCfg.setReadThrough(false);
> return ignite.getOrCreateCache(cCfg).withKeepBinary();
> }
>
> I have observed that method org.apache.ignite.spi.
> indexing.IndexingSpi#store
> is called for all keys, both primay and backup,
> resulting in both nodes having built duplicated index. In addition,
> org.apache.ignite.spi.indexing.IndexingSpi#remove is not called when the
> keys migrate to another node (for example due to second node joining the
> cluster).
> Is that by design or rather a bug? I would expect the receive remove()
> callback when primary keys migrate from the node, so I can free resources
> occupied by corresponding index.
>
> Thank you in advance for your help,
> zbyszek
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
--
Best regards,
Andrey V. Mashenkov