答复: 回复：BinaryObject and String.intern

[Shawn Du <sh...@neulion.com.cn>]

Hi  <ma...@gridgain.com> dkarachentsev,
 
Suppose I have a String array and array length is 1000. There are many duplicated values and only five distinct values in the array.
 
We store them using blow ways, which one save memory most?
#1 List<String>
#2 Map<String,BitSet>
In client side, #2 will save memory greatly, but how does it in ignite server?
If Ignite stores BitSet as Integer array, it seems that #2 will not save so much memory?
 
If there any tips to save memory for above case. Thanks in advance.
 
Thanks
Shawn
 
发件人: shawn.du [mailto:shawn.du@neulion.com.cn] 
发送时间: 2017年1月3日 18:37
收件人: user@ignite.apache.org
抄送: user@ignite.apache.org
主题: 回复：BinaryObject and String.intern
 
 
thanks  <ma...@gridgain.com> dkarachentsev.
 
在2017年01月03日 18:25， <ma...@gridgain.com> dkarachentsev 写道:
Actually no, because Ignite internally will store it as a BinaryObject and 
will send to other nodes in a binary format as well, where all string fields 
will be unmarshaled without intern(). 



-- 
View this message in context:  <http://apache-ignite-users.70518.x6.nabble.com/BinaryObject-and-String-intern-tp9826p9834.html> http://apache-ignite-users.70518.x6.nabble.com/BinaryObject-and-String-intern-tp9826p9834.html 
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: 答复: 回复：BinaryObject and String.intern

[dkarachentsev <dk...@gridgain.com>]

Hi Shawn,

BitSet will be marshaled using BitSet.writeObject() serialization
implementation. In other words, if this approach saves your memory in heap,
it will save it in Ignite as well.

If you save collection of strings, Ignite will not check for duplicates as
JVM does, and as mentioned Alexey, there is still no compression supported.
So only you can do is to write algorithms in your side to handle such cases
like your Map<String, BitSet>.

Thanks!



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/BinaryObject-and-String-intern-tp9826p9891.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: 答复: 回复：BinaryObject and String.intern

[Alexey Kuznetsov <ak...@apache.org>]

Hi, Shawn.

Some time ago I created issue "Provide some kind of pluggable compression
SPI support" [1]
But it is not implemented yet. May be in Ignie 2.0 there could be some kind
of such API.
See discussion [2]

[1] https://issues.apache.org/jira/browse/IGNITE-3592
[2]
http://apache-ignite-developers.2346864.n4.nabble.com/Data-compression-in-Ignite-2-0-td10099.html


On Thu, Jan 5, 2017 at 11:01 AM, Shawn Du <sh...@neulion.com.cn> wrote:

> Hi dkarachentsev <dk...@gridgain.com>,
>
>
>
> Suppose I have a String array and array length is 1000. There are many
> duplicated values and only five distinct values in the array.
>
>
>
> We store them using blow ways, which one save memory most?
>
> #1 List<String>
>
> #2 Map<String,BitSet>
>
> In client side, #2 will save memory greatly, but how does it in ignite
> server?
>
> If Ignite stores BitSet as Integer array, it seems that #2 will not save
> so much memory?
>
>
>
> If there any tips to save memory for above case. Thanks in advance.
>
>
>
> Thanks
>
> Shawn
>
>
>
> *发件人:* shawn.du [mailto:shawn.du@neulion.com.cn]
> *发送时间:* 2017年1月3日 18:37
> *收件人:* user@ignite.apache.org
> *抄送:* user@ignite.apache.org
> *主题:* 回复：BinaryObject and String.intern
>
>
>
>
>
> thanks dkarachentsev <dk...@gridgain.com>.
>
>
>
> 在2017年01月03日 18:25，dkarachentsev <dk...@gridgain.com> 写道:
>
> Actually no, because Ignite internally will store it as a BinaryObject and
> will send to other nodes in a binary format as well, where all string
> fields
> will be unmarshaled without intern().
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/BinaryObject-and-String-intern-tp9826p9834.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
>


-- 
Alexey Kuznetsov