You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Nikolay Izhikov <ni...@apache.org> on 2018/02/19 19:00:35 UTC
IgniteDataStreamer silently fails on a server node
Hello, Igniters.
While working on IGNITE-7727 I found strange behavior of IgniteDataStreamer:
If we have IgniteUuid as an indexed type update silently brokes on a server node.
Client doesn't have any notification about fails.
All calls of `addData`, `close`, etc. succeed on a client side but fails on server side.
I see 2 issue here:
1. The fail itself - it certainly a bug, I think I can fix it.
2. Lack of client notification. Is it OK when client doesn't know about fails of streamer updates?
Do we have this documented? I briefly looked at streamer docs but can't find description of such behavior.
Reproducer [1]
```
public void testStreamer() throws Exception {
Ignite client = grid("client");
CacheConfiguration ccfg = new CacheConfiguration("UUID_CACHE");
ccfg.setIndexedTypes(IgniteUuid.class, String.class);
client.createCache(ccfg);
try(IgniteDataStreamer<IgniteUuid, String> cache =
client.dataStreamer("UUID_CACHE")) {
for(Integer i=0; i<2; i++)
cache.addData(IgniteUuid.randomUuid().toString(), i.toString());
}
}
```
Server node stack trace [2]:
```
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to update index, incorrect key class [expCls=org.apache.ignite.lang.IgniteUuid,
actualCls=org.apache.ignite.internal.binary.BinaryObjectImpl]
at org.apache.ignite.internal.processors.query.GridQueryProcessor.typeByValue(GridQueryProcessor.java:1954)
at org.apache.ignite.internal.processors.query.GridQueryProcessor.store(GridQueryProcessor.java:1877)
at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.store(GridCacheQueryManager.java:403)
at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.finishUpdate(IgniteCacheOffheapManagerImpl.java:1343)
at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1207)
at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:345)
at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.storeValue(GridCacheMapEntry.java:3527)
at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.initialValue(GridCacheMapEntry.java:2735)
at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$IsolatedUpdater.receive(DataStreamerImpl.java:2113)
... 11 more
```
[1] https://gist.github.com/nizhikov/2e70a73c7b74a50fc89d270e9af1e1ca
[2] https://gist.github.com/nizhikov/c491c8f2b45aa59458b37b42b4b8dab4
Re: IgniteDataStreamer silently fails on a server node
Posted by Nikolay Izhikov <ni...@apache.org>.
> I set org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl#DFLT_MAX_REMAP_CNT to zero and see test fails now.
Anton, thank you very much!
I've prepared PR [1] to fix issue [2]. TC looks OK.
Looks like it was introduced by commit 53802d8b46f8ee09d51ba84267f65dc6f7e73a2f [3]
Can, anyone do review of my fix?
It literally consists of 1 LOC.
[1] https://github.com/apache/ignite/pull/3550
[2] https://issues.apache.org/jira/browse/IGNITE-7756
[3] https://github.com/SomeFire/ignite/commit/53802d8b46f8ee09d51ba84267f65dc6f7e73a2f
В Ср, 21/02/2018 в 15:49 +0300, Anton Vinogradov пишет:
> Nikolay,
>
> I checked your reproducer and it looks like retry helps data streamer to
> solve this.
> I
> set org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl#DFLT_MAX_REMAP_CNT
> to zero and see test fails now.
>
> Looks like second attempt is successful for such case.
>
> On Wed, Feb 21, 2018 at 1:36 PM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
>
> > On Wed, Feb 21, 2018 at 12:10 AM, Nikolay Izhikov <ni...@apache.org>
> > wrote:
> >
> > > Hello, Igniters.
> > >
> > > > Does this exception happen because IgniteUuid class is not correctly
> > > > handled?
> > >
> > > Yes. I've fixed it.
> > > But story doesn't end :)
> > >
> > > > As far as propagating exceptions, addData(...) is asynchronous
> >
> > operation
> > > > and returns IgniteFuture.
> > >
> > > Thank you, my bad.
> > >
> > > But, I can't understand when returned future should complete?
> > >
> > > Javadocs or documentation doesn't explain it.
> > > Also, I can't find any tests that checks future returned from addData.
> > >
> > > Can someone explan it to me?
> > >
> >
> > The returned future should complete either when the data is successfully
> > stored in cache, or whenever the failure occurs.
> >
Re: IgniteDataStreamer silently fails on a server node
Posted by Anton Vinogradov <av...@gridgain.com>.
Nikolay,
I checked your reproducer and it looks like retry helps data streamer to
solve this.
I
set org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl#DFLT_MAX_REMAP_CNT
to zero and see test fails now.
Looks like second attempt is successful for such case.
On Wed, Feb 21, 2018 at 1:36 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:
> On Wed, Feb 21, 2018 at 12:10 AM, Nikolay Izhikov <ni...@apache.org>
> wrote:
>
> > Hello, Igniters.
> >
> > > Does this exception happen because IgniteUuid class is not correctly
> > > handled?
> >
> > Yes. I've fixed it.
> > But story doesn't end :)
> >
> > > As far as propagating exceptions, addData(...) is asynchronous
> operation
> > > and returns IgniteFuture.
> >
> > Thank you, my bad.
> >
> > But, I can't understand when returned future should complete?
> >
> > Javadocs or documentation doesn't explain it.
> > Also, I can't find any tests that checks future returned from addData.
> >
> > Can someone explan it to me?
> >
>
> The returned future should complete either when the data is successfully
> stored in cache, or whenever the failure occurs.
>
Re: IgniteDataStreamer silently fails on a server node
Posted by Dmitriy Setrakyan <ds...@apache.org>.
On Wed, Feb 21, 2018 at 12:10 AM, Nikolay Izhikov <ni...@apache.org>
wrote:
> Hello, Igniters.
>
> > Does this exception happen because IgniteUuid class is not correctly
> > handled?
>
> Yes. I've fixed it.
> But story doesn't end :)
>
> > As far as propagating exceptions, addData(...) is asynchronous operation
> > and returns IgniteFuture.
>
> Thank you, my bad.
>
> But, I can't understand when returned future should complete?
>
> Javadocs or documentation doesn't explain it.
> Also, I can't find any tests that checks future returned from addData.
>
> Can someone explan it to me?
>
The returned future should complete either when the data is successfully
stored in cache, or whenever the failure occurs.
Re: IgniteDataStreamer silently fails on a server node
Posted by Nikolay Izhikov <ni...@apache.org>.
Hello, Igniters.
> Does this exception happen because IgniteUuid class is not correctly
> handled?
Yes. I've fixed it.
But story doesn't end :)
> As far as propagating exceptions, addData(...) is asynchronous operation
> and returns IgniteFuture.
Thank you, my bad.
But, I can't understand when returned future should complete?
Javadocs or documentation doesn't explain it.
Also, I can't find any tests that checks future returned from addData.
Can someone explan it to me?
В Пн, 19/02/2018 в 21:47 -0600, Dmitriy Setrakyan пишет:
> Nikolay,
>
> Does this exception happen because IgniteUuid class is not correctly
> handled? If that's the case, we should fix it. Would be great if you could
> do it.
>
> As far as propagating exceptions, addData(...) is asynchronous operation
> and returns IgniteFuture. The exception should be propagated to that
> future. Do you not see it there?
>
> D.
>
> On Mon, Feb 19, 2018 at 1:00 PM, Nikolay Izhikov <ni...@apache.org>
> wrote:
>
> > Hello, Igniters.
> >
> > While working on IGNITE-7727 I found strange behavior of
> > IgniteDataStreamer:
> >
> > If we have IgniteUuid as an indexed type update silently brokes on a
> > server node.
> > Client doesn't have any notification about fails.
> > All calls of `addData`, `close`, etc. succeed on a client side but fails
> > on server side.
> >
> > I see 2 issue here:
> >
> > 1. The fail itself - it certainly a bug, I think I can fix it.
> >
> > 2. Lack of client notification. Is it OK when client doesn't know about
> > fails of streamer updates?
> > Do we have this documented? I briefly looked at streamer docs but can't
> > find description of such behavior.
> >
> >
> > Reproducer [1]
> >
> > ```
> > public void testStreamer() throws Exception {
> > Ignite client = grid("client");
> >
> > CacheConfiguration ccfg = new CacheConfiguration("UUID_CACHE");
> >
> > ccfg.setIndexedTypes(IgniteUuid.class, String.class);
> >
> > client.createCache(ccfg);
> >
> > try(IgniteDataStreamer<IgniteUuid, String> cache =
> > client.dataStreamer("UUID_CACHE")) {
> >
> > for(Integer i=0; i<2; i++)
> > cache.addData(IgniteUuid.randomUuid().toString(),
> > i.toString());
> > }
> > }
> > ```
> >
> > Server node stack trace [2]:
> >
> > ```
> > Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
> > update index, incorrect key class [expCls=org.apache.ignite.
> > lang.IgniteUuid,
> > actualCls=org.apache.ignite.internal.binary.BinaryObjectImpl]
> > at org.apache.ignite.internal.processors.query.GridQueryProcessor.
> > typeByValue(GridQueryProcessor.java:1954)
> > at org.apache.ignite.internal.processors.query.
> > GridQueryProcessor.store(GridQueryProcessor.java:1877)
> > at org.apache.ignite.internal.processors.cache.query.
> > GridCacheQueryManager.store(GridCacheQueryManager.java:403)
> > at org.apache.ignite.internal.processors.cache.
> > IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.finishUpdate(
> > IgniteCacheOffheapManagerImpl.java:1343)
> > at org.apache.ignite.internal.processors.cache.
> > IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(
> > IgniteCacheOffheapManagerImpl.java:1207)
> > at org.apache.ignite.internal.processors.cache.
> > IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.
> > java:345)
> > at org.apache.ignite.internal.processors.cache.
> > GridCacheMapEntry.storeValue(GridCacheMapEntry.java:3527)
> > at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.
> > initialValue(GridCacheMapEntry.java:2735)
> > at org.apache.ignite.internal.processors.datastreamer.
> > DataStreamerImpl$IsolatedUpdater.receive(DataStreamerImpl.java:2113)
> > ... 11 more
> > ```
> >
> > [1] https://gist.github.com/nizhikov/2e70a73c7b74a50fc89d270e9af1e1ca
> >
> > [2] https://gist.github.com/nizhikov/c491c8f2b45aa59458b37b42b4b8dab4
Re: IgniteDataStreamer silently fails on a server node
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Nikolay,
Does this exception happen because IgniteUuid class is not correctly
handled? If that's the case, we should fix it. Would be great if you could
do it.
As far as propagating exceptions, addData(...) is asynchronous operation
and returns IgniteFuture. The exception should be propagated to that
future. Do you not see it there?
D.
On Mon, Feb 19, 2018 at 1:00 PM, Nikolay Izhikov <ni...@apache.org>
wrote:
> Hello, Igniters.
>
> While working on IGNITE-7727 I found strange behavior of
> IgniteDataStreamer:
>
> If we have IgniteUuid as an indexed type update silently brokes on a
> server node.
> Client doesn't have any notification about fails.
> All calls of `addData`, `close`, etc. succeed on a client side but fails
> on server side.
>
> I see 2 issue here:
>
> 1. The fail itself - it certainly a bug, I think I can fix it.
>
> 2. Lack of client notification. Is it OK when client doesn't know about
> fails of streamer updates?
> Do we have this documented? I briefly looked at streamer docs but can't
> find description of such behavior.
>
>
> Reproducer [1]
>
> ```
> public void testStreamer() throws Exception {
> Ignite client = grid("client");
>
> CacheConfiguration ccfg = new CacheConfiguration("UUID_CACHE");
>
> ccfg.setIndexedTypes(IgniteUuid.class, String.class);
>
> client.createCache(ccfg);
>
> try(IgniteDataStreamer<IgniteUuid, String> cache =
> client.dataStreamer("UUID_CACHE")) {
>
> for(Integer i=0; i<2; i++)
> cache.addData(IgniteUuid.randomUuid().toString(),
> i.toString());
> }
> }
> ```
>
> Server node stack trace [2]:
>
> ```
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
> update index, incorrect key class [expCls=org.apache.ignite.
> lang.IgniteUuid,
> actualCls=org.apache.ignite.internal.binary.BinaryObjectImpl]
> at org.apache.ignite.internal.processors.query.GridQueryProcessor.
> typeByValue(GridQueryProcessor.java:1954)
> at org.apache.ignite.internal.processors.query.
> GridQueryProcessor.store(GridQueryProcessor.java:1877)
> at org.apache.ignite.internal.processors.cache.query.
> GridCacheQueryManager.store(GridCacheQueryManager.java:403)
> at org.apache.ignite.internal.processors.cache.
> IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.finishUpdate(
> IgniteCacheOffheapManagerImpl.java:1343)
> at org.apache.ignite.internal.processors.cache.
> IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(
> IgniteCacheOffheapManagerImpl.java:1207)
> at org.apache.ignite.internal.processors.cache.
> IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.
> java:345)
> at org.apache.ignite.internal.processors.cache.
> GridCacheMapEntry.storeValue(GridCacheMapEntry.java:3527)
> at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.
> initialValue(GridCacheMapEntry.java:2735)
> at org.apache.ignite.internal.processors.datastreamer.
> DataStreamerImpl$IsolatedUpdater.receive(DataStreamerImpl.java:2113)
> ... 11 more
> ```
>
> [1] https://gist.github.com/nizhikov/2e70a73c7b74a50fc89d270e9af1e1ca
>
> [2] https://gist.github.com/nizhikov/c491c8f2b45aa59458b37b42b4b8dab4