Efficient way to get partial data of a cache entry

[Shawn Du <sh...@neulion.com.cn>]

Hi,

 

Given a key, how to get partial data of a cache entry efficiently?

 

#1 by SQL (SELECT field, . FROM t where _key='key')

#2 by cache.get(key) directly

#3 get binaryObject by cache.get(key) and read by field.

 

Thanks

Shawn

Re: Efficient way to get partial data of a cache entry

[dkarachentsev <dk...@gridgain.com>]

Probably 1 and 3 would be the most effective ways if you're getting objects
on affinity node [1], not use CacheMemoryMode#OFFHEAP_VALUES mode and
CacheConfiguration.setCopyOnRead() is set to false (for query also with
Query.setLocal() set to true).

But copyOnRead = false flag could be the cause of errors, because result is
got directly from cache, so the compromise option, I think, would be #1.
Safe and with not a big overhead.

[1] Affinity node means the node where resides key/value data.



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Efficient-way-to-get-partial-data-of-a-cache-entry-tp9965p9969.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: 答复: 答复: Efficient way to get partial data of a cache entry

[vkulichenko <va...@gmail.com>]

What is the exception? Can you show the trace?

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Efficient-way-to-get-partial-data-of-a-cache-entry-tp9965p10047.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

答复: 答复: Efficient way to get partial data of a cache entry

[Shawn Du <sh...@neulion.com.cn>]

Hi Val,

Thanks for your help. It works. But throw new issues.

RoaringBitmap bitmap = new RoaringBitmap();
bitmap.add(100);
BinaryObject bo = ignite.binary().toBinary(bitmap); // exception throw here.
Ignite can't build binary format for RoaringBitmap. I test java class, it
works. 

Please help. 
Thanks Shawn.

-----邮件原件-----
发件人: vkulichenko [mailto:valentin.kulichenko@gmail.com] 
发送时间: 2017年1月12日 5:42
收件人: user@ignite.apache.org
主题: Re: 答复: Efficient way to get partial data of a cache entry

Shawn,

BinaryObject always return another BinaryObject for your custom type,
because otherwise it would mean that you have classes and most likely there
is no reason to use BinaryObject and withKeepBinary in the first place.
However, you can always call BinaryObject.deserialize() to convert to
instance of your class.

-Val



--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Efficient-way-to-get-partial-
data-of-a-cache-entry-tp9965p10042.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: 答复: Efficient way to get partial data of a cache entry

[vkulichenko <va...@gmail.com>]

Shawn,

BinaryObject always return another BinaryObject for your custom type,
because otherwise it would mean that you have classes and most likely there
is no reason to use BinaryObject and withKeepBinary in the first place.
However, you can always call BinaryObject.deserialize() to convert to
instance of your class.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Efficient-way-to-get-partial-data-of-a-cache-entry-tp9965p10042.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

答复: Efficient way to get partial data of a cache entry

[Shawn Du <sh...@neulion.com.cn>]

 

Hi,

 

It seems that we can’t read  customized class from BinaryObject by calling BinaryObject.field() or FieldType.value.

They will both return BinaryObjectImpl which can’t cast to my customized class. 

 

I also see this javadoc for withKeepBinary. So isn’t it? This is a limitation? 

 

/**
* Returns cache that will operate with binary objects.
* <p>
* Cache returned by this method will not be forced to deserialize binary objects,
* so keys and values will be returned from cache API methods without changes. Therefore,
* signature of the cache can contain only following types:
* <ul>
*     <li><code>org.apache.ignite.binary.BinaryObject</code> for binary classes</li>
*     <li>All primitives (byte, int, ...) and there boxed versions (Byte, Integer, ...)</li>
*     <li>Arrays of primitives (byte[], int[], ...)</li>
*     <li>{@link String} and array of {@link String}s</li>
*     <li>{@link UUID} and array of {@link UUID}s</li>
*     <li>{@link Date} and array of {@link Date}s</li>
*     <li>{@link Timestamp} and array of {@link Timestamp}s</li>
*     <li>Enums and array of enums</li>
*     <li>
*         Maps, collections and array of objects (but objects inside
*         them will still be converted if they are binary)
*     </li>
* </ul>
* <p>

 

Thanks

Shawn

 

发件人: Shawn Du [mailto:shawn.du@neulion.com.cn] 
发送时间: 2017年1月10日 8:36
收件人: user@ignite.apache.org
主题: 答复: Efficient way to get partial data of a cache entry

 

Thanks all’s reply.

 

It seems that we all agree that #2 is not the answer. And Yakov give new answers. I will have a test all of them and share the result here. 

 

 

Thanks

Shawn

 

发件人: Yakov Zhdanov [mailto:yzhdanov@apache.org] 
发送时间: 2017年1月9日 19:21
收件人: user@ignite.apache.org <ma...@ignite.apache.org> 
主题: Re: Efficient way to get partial data of a cache entry

 

I would suggest calling cache.invoke() and return needed result without altering the entry.

 

Or compute.affinityCall() and do local get inside callable and computing and returning the result.

答复: Efficient way to get partial data of a cache entry

[Shawn Du <sh...@neulion.com.cn>]

Thanks all’s reply.

 

It seems that we all agree that #2 is not the answer. And Yakov give new answers. I will have a test all of them and share the result here. 

 

 

Thanks

Shawn

 

发件人: Yakov Zhdanov [mailto:yzhdanov@apache.org] 
发送时间: 2017年1月9日 19:21
收件人: user@ignite.apache.org
主题: Re: Efficient way to get partial data of a cache entry

 

I would suggest calling cache.invoke() and return needed result without altering the entry.

 

Or compute.affinityCall() and do local get inside callable and computing and returning the result.

Re: Efficient way to get partial data of a cache entry

[Yakov Zhdanov <yz...@apache.org>]

I would suggest calling cache.invoke() and return needed result without
altering the entry.

Or compute.affinityCall() and do local get inside callable and computing
and returning the result.

Re: Efficient way to get partial data of a cache entry

[Andrey Mashenkov <an...@gmail.com>]

Hi Shawn,


#1 way is most unefficient way to get entry by the known key, due to SQL
parsing and planning overhead.
#2 way can cause unwanted deserialization in case of big value objects.
#3 way look like the most efficient way if you need the only field or few
fields of big value object.

RE: Efficient way to get partial data of a cache entry

[vkulichenko <va...@gmail.com>]

RoaringBitmap is Externalizable with custom serialization logic, therefore
can't be represented as BinaryObject. In such cases toBinary() method return
the original object without changes.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Efficient-way-to-get-partial-data-of-a-cache-entry-tp9965p10051.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.