You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by "naresh.goty" <gn...@gmail.com> on 2017/10/28 05:57:48 UTC
Exception in calculating cache size
Hi All,
We are seeing an issue with calculating size of few caches in our
application, only when there
are more than one server node is started at the same time.
As part of startup routine, we have a custom load routine to load data to
cache, and once load is successfully complete, then cache size is
calculated.
Below is the error observed in the log when the size() method is invoked:
{"eventTime" : "Oct 20 2017 10:50:25.698 AM PDT", "thread" :
"CacheExecutor-3", "level" : "ERROR", "class" : "com.service.CacheManager",
"method" : "lambda$doLoad$146", "message" : ""}
java.lang.ClassCastException: org.apache.ignite.lang.IgniteBiTuple cannot be
cast to java.lang.Integer
at
java.util.stream.ReferencePipeline$4$1.accept(ReferencePipeline.java:210)
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at
java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.IntPipeline.reduce(IntPipeline.java:456)
at java.util.stream.IntPipeline.sum(IntPipeline.java:414)
at com.cache.RangeCache.size(RangeCache.java:369)
public int size() {
return((IgniteCache<Object, Entity>)cache)
.query(new ScanQuery<Object, Entity>(), (entry) ->
((CompositeEntity)entry.getValue()).size())
.getAll()
.stream()
.mapToInt(Integer::intValue)
.sum();
}
Note: Entity is a custom interface for cache entities, and CompositeEntity
implements Entity.
Please let us know if there if you find any issue with above method of
calculating cache size.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Exception in calculating cache size
Posted by "naresh.goty" <gn...@gmail.com>.
Thanks much Andrey & Dmitry, the solution is working.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Exception in calculating cache size
Posted by Andrey Mashenkov <an...@gmail.com>.
Dmitry,
Yes, you are right. we should broadcast a job with local scan queries.
On Mon, Oct 30, 2017 at 4:54 PM, Dmitry Pavlov <dp...@gmail.com>
wrote:
> Hi Andrey, should we limit scan query to be executed only for current node
> here with setLocal() method?
>
>
> пн, 30 окт. 2017 г. в 16:48, Andrey Mashenkov <andrey.mashenkov@gmail.com
> >:
>
>> As a workaround you can try to broadcast a task :
>>
>> Collection<Integer> res = ignite.compute(ignite.cluster(
>> ).forCacheNodes("mycache")).broadcast(
>>
>> new IgniteCallable<Integer>() {
>> /** Auto-inject ignite instance. */
>> @IgniteInstanceResource
>> private Ignite ignite;
>>
>> @Override public Integer call() {
>> IgniteCache<Integer, Integer> cache =
>> ignite.getOrCreateCache(CACHE_NAME);
>> Iterator<Cache.Entry> curs = ((IgniteCache<Object,
>> Entity>)cache)
>> .query(new ScanQuery<Object, Entity>()
>> ).iterator();
>>
>> int cnt=0;
>> while(curs.hasNext()){
>> cnt += ((CompositeEntity)entry.get
>> Value()).size()
>> }
>> return cnt;
>> }
>> }
>> );
>>
>> On Mon, Oct 30, 2017 at 4:38 PM, Andrey Mashenkov <
>> andrey.mashenkov@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> This is known bug that transformer applies only to first page of results.
>>> Here is a ticket [1]
>>>
>>>
>>> [1] https://issues.apache.org/jira/browse/IGNITE-5804
>>>
>>> On Sat, Oct 28, 2017 at 8:57 AM, naresh.goty <gn...@gmail.com>
>>> wrote:
>>>
>>>> Hi All,
>>>>
>>>> We are seeing an issue with calculating size of few caches in our
>>>> application, only when there
>>>> are more than one server node is started at the same time.
>>>> As part of startup routine, we have a custom load routine to load data
>>>> to
>>>> cache, and once load is successfully complete, then cache size is
>>>> calculated.
>>>>
>>>> Below is the error observed in the log when the size() method is
>>>> invoked:
>>>>
>>>> {"eventTime" : "Oct 20 2017 10:50:25.698 AM PDT", "thread" :
>>>> "CacheExecutor-3", "level" : "ERROR", "class" :
>>>> "com.service.CacheManager",
>>>> "method" : "lambda$doLoad$146", "message" : ""}
>>>> java.lang.ClassCastException: org.apache.ignite.lang.IgniteBiTuple
>>>> cannot be
>>>> cast to java.lang.Integer
>>>> at
>>>> java.util.stream.ReferencePipeline$4$1.accept(
>>>> ReferencePipeline.java:210)
>>>> at
>>>> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.
>>>> java:1374)
>>>> at java.util.stream.AbstractPipeline.copyInto(
>>>> AbstractPipeline.java:481)
>>>> at
>>>> java.util.stream.AbstractPipeline.wrapAndCopyInto(
>>>> AbstractPipeline.java:471)
>>>> at
>>>> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(
>>>> ReduceOps.java:708)
>>>> at java.util.stream.AbstractPipeline.evaluate(
>>>> AbstractPipeline.java:234)
>>>> at java.util.stream.IntPipeline.reduce(IntPipeline.java:456)
>>>> at java.util.stream.IntPipeline.sum(IntPipeline.java:414)
>>>> at com.cache.RangeCache.size(RangeCache.java:369)
>>>>
>>>>
>>>>
>>>> public int size() {
>>>>
>>>> return((IgniteCache<Object, Entity>)cache)
>>>> .query(new ScanQuery<Object,
>>>> Entity>(), (entry) ->
>>>> ((CompositeEntity)entry.getValue()).size())
>>>> .getAll()
>>>> .stream()
>>>> .mapToInt(Integer::intValue)
>>>> .sum();
>>>>
>>>> }
>>>> Note: Entity is a custom interface for cache entities, and
>>>> CompositeEntity
>>>> implements Entity.
>>>>
>>>> Please let us know if there if you find any issue with above
>>>> method of
>>>> calculating cache size.
>>>>
>>>>
>>>>
>>>> --
>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Andrey V. Mashenkov
>>>
>>
>>
>>
>> --
>> Best regards,
>> Andrey V. Mashenkov
>>
>
--
Best regards,
Andrey V. Mashenkov
Re: Exception in calculating cache size
Posted by Dmitry Pavlov <dp...@gmail.com>.
Hi Andrey, should we limit scan query to be executed only for current node
here with setLocal() method?
пн, 30 окт. 2017 г. в 16:48, Andrey Mashenkov <an...@gmail.com>:
> As a workaround you can try to broadcast a task :
>
> Collection<Integer> res =
> ignite.compute(ignite.cluster().forCacheNodes("mycache")).broadcast(
>
> new IgniteCallable<Integer>() {
> /** Auto-inject ignite instance. */
> @IgniteInstanceResource
> private Ignite ignite;
>
> @Override public Integer call() {
> IgniteCache<Integer, Integer> cache =
> ignite.getOrCreateCache(CACHE_NAME);
> Iterator<Cache.Entry> curs = ((IgniteCache<Object,
> Entity>)cache)
> .query(new ScanQuery<Object, Entity>()
> ).iterator();
>
> int cnt=0;
> while(curs.hasNext()){
> cnt += ((CompositeEntity)entry.getValue()).size()
>
> }
> return cnt;
> }
> }
> );
>
> On Mon, Oct 30, 2017 at 4:38 PM, Andrey Mashenkov <
> andrey.mashenkov@gmail.com> wrote:
>
>> Hi,
>>
>> This is known bug that transformer applies only to first page of results.
>> Here is a ticket [1]
>>
>>
>> [1] https://issues.apache.org/jira/browse/IGNITE-5804
>>
>> On Sat, Oct 28, 2017 at 8:57 AM, naresh.goty <gn...@gmail.com>
>> wrote:
>>
>>> Hi All,
>>>
>>> We are seeing an issue with calculating size of few caches in our
>>> application, only when there
>>> are more than one server node is started at the same time.
>>> As part of startup routine, we have a custom load routine to load data to
>>> cache, and once load is successfully complete, then cache size is
>>> calculated.
>>>
>>> Below is the error observed in the log when the size() method is invoked:
>>>
>>> {"eventTime" : "Oct 20 2017 10:50:25.698 AM PDT", "thread" :
>>> "CacheExecutor-3", "level" : "ERROR", "class" :
>>> "com.service.CacheManager",
>>> "method" : "lambda$doLoad$146", "message" : ""}
>>> java.lang.ClassCastException: org.apache.ignite.lang.IgniteBiTuple
>>> cannot be
>>> cast to java.lang.Integer
>>> at
>>> java.util.stream.ReferencePipeline$4$1.accept(ReferencePipeline.java:210)
>>> at
>>>
>>> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
>>> at
>>> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
>>> at
>>>
>>> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
>>> at
>>>
>>> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
>>> at
>>> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
>>> at java.util.stream.IntPipeline.reduce(IntPipeline.java:456)
>>> at java.util.stream.IntPipeline.sum(IntPipeline.java:414)
>>> at com.cache.RangeCache.size(RangeCache.java:369)
>>>
>>>
>>>
>>> public int size() {
>>>
>>> return((IgniteCache<Object, Entity>)cache)
>>> .query(new ScanQuery<Object,
>>> Entity>(), (entry) ->
>>> ((CompositeEntity)entry.getValue()).size())
>>> .getAll()
>>> .stream()
>>> .mapToInt(Integer::intValue)
>>> .sum();
>>>
>>> }
>>> Note: Entity is a custom interface for cache entities, and
>>> CompositeEntity
>>> implements Entity.
>>>
>>> Please let us know if there if you find any issue with above
>>> method of
>>> calculating cache size.
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>
>>
>>
>>
>> --
>> Best regards,
>> Andrey V. Mashenkov
>>
>
>
>
> --
> Best regards,
> Andrey V. Mashenkov
>
Re: Exception in calculating cache size
Posted by Andrey Mashenkov <an...@gmail.com>.
As a workaround you can try to broadcast a task :
Collection<Integer> res =
ignite.compute(ignite.cluster().forCacheNodes("mycache")).broadcast(
new IgniteCallable<Integer>() {
/** Auto-inject ignite instance. */
@IgniteInstanceResource
private Ignite ignite;
@Override public Integer call() {
IgniteCache<Integer, Integer> cache =
ignite.getOrCreateCache(CACHE_NAME);
Iterator<Cache.Entry> curs = ((IgniteCache<Object,
Entity>)cache)
.query(new ScanQuery<Object, Entity>()
).iterator();
int cnt=0;
while(curs.hasNext()){
cnt += ((CompositeEntity)entry.getValue()).size()
}
return cnt;
}
}
);
On Mon, Oct 30, 2017 at 4:38 PM, Andrey Mashenkov <
andrey.mashenkov@gmail.com> wrote:
> Hi,
>
> This is known bug that transformer applies only to first page of results.
> Here is a ticket [1]
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-5804
>
> On Sat, Oct 28, 2017 at 8:57 AM, naresh.goty <gn...@gmail.com>
> wrote:
>
>> Hi All,
>>
>> We are seeing an issue with calculating size of few caches in our
>> application, only when there
>> are more than one server node is started at the same time.
>> As part of startup routine, we have a custom load routine to load data to
>> cache, and once load is successfully complete, then cache size is
>> calculated.
>>
>> Below is the error observed in the log when the size() method is invoked:
>>
>> {"eventTime" : "Oct 20 2017 10:50:25.698 AM PDT", "thread" :
>> "CacheExecutor-3", "level" : "ERROR", "class" :
>> "com.service.CacheManager",
>> "method" : "lambda$doLoad$146", "message" : ""}
>> java.lang.ClassCastException: org.apache.ignite.lang.IgniteBiTuple
>> cannot be
>> cast to java.lang.Integer
>> at
>> java.util.stream.ReferencePipeline$4$1.accept(ReferencePipeline.java:210)
>> at
>> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(
>> ArrayList.java:1374)
>> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.
>> java:481)
>> at
>> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPi
>> peline.java:471)
>> at
>> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Reduc
>> eOps.java:708)
>> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.
>> java:234)
>> at java.util.stream.IntPipeline.reduce(IntPipeline.java:456)
>> at java.util.stream.IntPipeline.sum(IntPipeline.java:414)
>> at com.cache.RangeCache.size(RangeCache.java:369)
>>
>>
>>
>> public int size() {
>>
>> return((IgniteCache<Object, Entity>)cache)
>> .query(new ScanQuery<Object, Entity>(),
>> (entry) ->
>> ((CompositeEntity)entry.getValue()).size())
>> .getAll()
>> .stream()
>> .mapToInt(Integer::intValue)
>> .sum();
>>
>> }
>> Note: Entity is a custom interface for cache entities, and
>> CompositeEntity
>> implements Entity.
>>
>> Please let us know if there if you find any issue with above
>> method of
>> calculating cache size.
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
>
>
> --
> Best regards,
> Andrey V. Mashenkov
>
--
Best regards,
Andrey V. Mashenkov
Re: Exception in calculating cache size
Posted by Andrey Mashenkov <an...@gmail.com>.
Hi,
This is known bug that transformer applies only to first page of results.
Here is a ticket [1]
[1] https://issues.apache.org/jira/browse/IGNITE-5804
On Sat, Oct 28, 2017 at 8:57 AM, naresh.goty <gn...@gmail.com> wrote:
> Hi All,
>
> We are seeing an issue with calculating size of few caches in our
> application, only when there
> are more than one server node is started at the same time.
> As part of startup routine, we have a custom load routine to load data to
> cache, and once load is successfully complete, then cache size is
> calculated.
>
> Below is the error observed in the log when the size() method is invoked:
>
> {"eventTime" : "Oct 20 2017 10:50:25.698 AM PDT", "thread" :
> "CacheExecutor-3", "level" : "ERROR", "class" : "com.service.CacheManager",
> "method" : "lambda$doLoad$146", "message" : ""}
> java.lang.ClassCastException: org.apache.ignite.lang.IgniteBiTuple cannot
> be
> cast to java.lang.Integer
> at
> java.util.stream.ReferencePipeline$4$1.accept(ReferencePipeline.java:210)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.
> java:1374)
> at java.util.stream.AbstractPipeline.copyInto(
> AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(
> AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(
> AbstractPipeline.java:234)
> at java.util.stream.IntPipeline.reduce(IntPipeline.java:456)
> at java.util.stream.IntPipeline.sum(IntPipeline.java:414)
> at com.cache.RangeCache.size(RangeCache.java:369)
>
>
>
> public int size() {
>
> return((IgniteCache<Object, Entity>)cache)
> .query(new ScanQuery<Object, Entity>(),
> (entry) ->
> ((CompositeEntity)entry.getValue()).size())
> .getAll()
> .stream()
> .mapToInt(Integer::intValue)
> .sum();
>
> }
> Note: Entity is a custom interface for cache entities, and
> CompositeEntity
> implements Entity.
>
> Please let us know if there if you find any issue with above
> method of
> calculating cache size.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
--
Best regards,
Andrey V. Mashenkov