You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by guill-melo <gu...@gmail.com> on 2016/05/23 21:05:21 UTC
Working Directly with Binary objects
Hello All,
I have just recently started using Ignite, and I am really enjoying it, I am
trying to profile the size of some objects and compare it with other types
of serializations, is it possible to use the serialization without starting
a Ignite process? The only way I could do it was:
IgniteConfiguration cfg = new IgniteConfiguration();
Ignite ignite = Ignition.start(cfg);
Entity entity = Entity
.build()
.withId("12345")
.withField("value field")
.withOtherField(33)
.build();
BinaryObject binaryObject = ignite.binary().toBinary(entity);
Also, Is there a utility to convert the BinaryObject to byte[] ou should I
just treat it like any other object?
Thanks !!
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Working-Directly-with-Binary-objects-tp5131.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Working Directly with Binary objects
Posted by guill-melo <gu...@gmail.com>.
Thank you very much Denis, will look into that.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Working-Directly-with-Binary-objects-tp5131p5331.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Working Directly with Binary objects
Posted by Denis Magda <dm...@gridgain.com>.
Hi,
Actually it’s not a difficult task to write a cache entry in a byte representation into a storage if it’s needed.
The easiest way is just to take an instance of a marshaller that is globally used (ignite.configuration().getMarshaller()) and convert a value and a key of an entry passed to CacheStore.write(Cache.Entry<? extends K, ? extends V> entry) with a call like marshaller.marshal(val.getValue()).
What is more interesting is to implement a CacheStore for some other persistent store. You can refer to existed implementations of
CacheJdbcBlobStore and CassandraCacheStore.
—
Denis
> On May 25, 2016, at 5:50 PM, guill-melo <gu...@gmail.com> wrote:
>
> Hello Denis,
> I am not trying to add this to production, I am investigating the
> feasibility of having a CacheStore that writes to HBase or something similar
> in a byte[] representation of the BynaryObject, as it is more efficient than
> java.io.
>
> I am trying to see what I can do with the API, as I am very new to it and
> am quite excited about it.
>
> Thank you Denis
>
>
>
> --
> View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Working-Directly-with-Binary-objects-tp5131p5197.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Working Directly with Binary objects
Posted by guill-melo <gu...@gmail.com>.
Hello Denis,
I am not trying to add this to production, I am investigating the
feasibility of having a CacheStore that writes to HBase or something similar
in a byte[] representation of the BynaryObject, as it is more efficient than
java.io.
I am trying to see what I can do with the API, as I am very new to it and
am quite excited about it.
Thank you Denis
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Working-Directly-with-Binary-objects-tp5131p5197.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Working Directly with Binary objects
Posted by Denis Magda <dm...@gridgain.com>.
Hi,
What is your use case and where do you want to apply the techniques
suggested in this discussion?
Actually, I would highly discourage you from using this code in production
because it's based on private Ignite API that is used by Ignite developers
only.
--
Denis
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Working-Directly-with-Binary-objects-tp5131p5185.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Working Directly with Binary objects
Posted by guill-melo <gu...@gmail.com>.
Hello Alexei, Thanks again for your time,
by a lot more compact I mean that a object like:
Entity entity = Entity
.build()
.withId("12345")
.withField("value field")
.withOtherField(33)
.build();
using java.io serialization is 237 bytes, while a BinaryObject is only 58,
so thats what I ment by "more compact" .
I am trying to create a CacheStore that stores the objects as binaries,
possibly using HBase, or something else that stores byte[].
This is mostly investigating, as I am very curious to create a cache with a
'big data' backing store, and though this would be a good opportunity to
improve my Ignite knowledge, I came from the Coherence world.
Hope that makes a little more sense. I don't want to store the objects in
the cache as byte[], only in the CacheStore, and the data should only be
accessed though the cache.
Thanks.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Working-Directly-with-Binary-objects-tp5131p5196.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Working Directly with Binary objects
Posted by Alexei Scherbakov <al...@gmail.com>.
Hi,
You can marshall and unmarshall BinaryObject using marshaller like any
other type.
BTW, what are you trying to achieve ?
If you store cache values as byte arrays, you need to deserialize them on
each data access to do something useful.
And how do you conclude "as its a lot core compact" ?
2016-05-24 19:47 GMT+03:00 guill-melo <gu...@gmail.com>:
> Hello Alexei,
> Thank you very much for the reply, that really helped me.
>
> Just a quick follow up, how can I create a BinaryObject from a byte[]? I
> playing with a BinaryCacheStore and I want to persist the objects in the
> serialized BinaryObject format, as its a lot core compact.
>
> Thanks again.
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Working-Directly-with-Binary-objects-tp5131p5152.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
--
Best regards,
Alexei Scherbakov
Re: Working Directly with Binary objects
Posted by guill-melo <gu...@gmail.com>.
Hello Alexei,
Thank you very much for the reply, that really helped me.
Just a quick follow up, how can I create a BinaryObject from a byte[]? I
playing with a BinaryCacheStore and I want to persist the objects in the
serialized BinaryObject format, as its a lot core compact.
Thanks again.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Working-Directly-with-Binary-objects-tp5131p5152.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Working Directly with Binary objects
Posted by Alexei Scherbakov <al...@gmail.com>.
Hi,
try something like:
String dir = System.getProperty("java.io.tmpdir");
IgniteUtils.setWorkDirectory(dir, null);
IgniteConfiguration iCfg = new IgniteConfiguration();
BinaryConfiguration bCfg = new BinaryConfiguration();
iCfg.setBinaryConfiguration(bCfg);
BinaryContext ctx = new
BinaryContext(BinaryCachingMetadataHandler.create(), iCfg, new
NullLogger());
BinaryMarshaller marsh = new BinaryMarshaller();
marsh.setContext(new MarshallerContextImpl(null));
IgniteUtils.invoke(BinaryMarshaller.class, marsh, "setBinaryContext",
ctx, iCfg);
byte[] bytes = marsh.marshal(new HashMap<String, String>() {
{
put("1", "1");
}
});
For second question:
((BinaryObjectImpl)bo).array()
2016-05-24 0:05 GMT+03:00 guill-melo <gu...@gmail.com>:
> Hello All,
> I have just recently started using Ignite, and I am really enjoying it, I
> am
> trying to profile the size of some objects and compare it with other types
> of serializations, is it possible to use the serialization without starting
> a Ignite process? The only way I could do it was:
>
> IgniteConfiguration cfg = new IgniteConfiguration();
> Ignite ignite = Ignition.start(cfg);
> Entity entity = Entity
> .build()
> .withId("12345")
> .withField("value field")
> .withOtherField(33)
> .build();
> BinaryObject binaryObject = ignite.binary().toBinary(entity);
>
> Also, Is there a utility to convert the BinaryObject to byte[] ou should I
> just treat it like any other object?
>
> Thanks !!
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Working-Directly-with-Binary-objects-tp5131.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
--
Best regards,
Alexei Scherbakov