Store raw binary value in Apache Ignite through thin python client

[kulinskyvs <ku...@gmail.com>]

I'm trying to save some raw binary data into Apache Ignite using thin Python
client and the process is very slow (actually it looks like it depends on
the size of the data).

For my simple test case I've started locally Ignite with single node
(version 2.7.0 without any specific configuration):


Also, I have a binary file with the content I want to store in Ignite. The
size if around 6MB. Here is my source code:


After *my_cache.put *the process just freezes. I haven't even managed to
wait till the end (I've terminated it after 2 minutes).

However, passing the file content transformed into a string works very fast
.


What I'm missing?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Store raw binary value in Apache Ignite through thin python client

[Evgeniy Ignatiev <ye...@gmail.com>]

Hi.

I think the issue is related to building bytes counterpart of an array 
during serialization using += operation on bytes object which is 
immutable. This leads to throwing away previously created bytes 
instances, each of which grows at new iteration.

This aligns with what I observe when run the test provided here.

Quick fix - 
https://github.com/YevIgn/ignite/commit/64211b2d7c9e174f0c8979cbb4f26df52b8ebcd8

After it, the array building stops hanging on my machine.

Should I open PR?

Note: First saw the issue, the message here is copy from my comment in it.

Best regards,
Evgeniy Ignatiev.

On 5/16/2019 4:42 PM, kulinskyvs wrote:
> Hi,
>
> Thank you for your feedback!
> Just for information - I've also created a stackoverflow question
> (https://stackoverflow.com/questions/56127565/store-raw-binary-value-in-apache-ignite-through-thin-python-client)
> related to this topic > I'll post the JIRA issue link there.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Store raw binary value in Apache Ignite through thin python client

[kulinskyvs <ku...@gmail.com>]

Hi,

Thank you for your feedback!
Just for information - I've also created a stackoverflow question
(https://stackoverflow.com/questions/56127565/store-raw-binary-value-in-apache-ignite-through-thin-python-client)
related to this topic > I'll post the JIRA issue link there.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Store raw binary value in Apache Ignite through thin python client

[Denis Mekhanikov <dm...@gmail.com>]

Hi!

Thanks for the report! Seems like the implementation of serialization of
primitive arrays is not optimal.
I created a JIRA ticket for this issue:
https://issues.apache.org/jira/browse/IGNITE-11854

Denis

вт, 14 мая 2019 г. в 13:53, kulinskyvs <ku...@gmail.com>:

> I'm trying to save some raw binary data into Apache Ignite using thin
> Python
> client and the process is very slow (actually it looks like it depends on
> the size of the data).
>
> For my simple test case I've started locally Ignite with single node
> (version 2.7.0 without any specific configuration):
>
>
> Also, I have a binary file with the content I want to store in Ignite. The
> size if around 6MB. Here is my source code:
>
>
> After *my_cache.put *the process just freezes. I haven't even managed to
> wait till the end (I've terminated it after 2 minutes).
>
> However, passing the file content transformed into a string works very fast
> .
>
>
> What I'm missing?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>