You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Stepan Pilschikov <pi...@gmail.com> on 2018/10/26 13:57:08 UTC
Thin clients all in one
Hi, everyone
Create new thread to centralize cross compatibility and others common
problems between thin clients
Tying to use Object array to exchange different data between JS, PHP and
Python thin clients
JS and PHP simply can't put any type of arrays
Python can put data, but if you take it, data would be completely
different, maybe during this put process data is changed
Code and output:
PHP - https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
JS - https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
Python -
https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
I'm tried different data types (string, double, float, complex objects,
just random objects, char, byte, Date), still
How, from my perspective, it should works:
put array of any type and then get this array.
Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new Date()] -> get
[[Date object], [Date object]] (like in java thin client)
Looks like bug in all clients, what you think?
I wanted to try Collections, but i think this type have same problem
Re: Thin clients all in one
Posted by Stepan Pilschikov <pi...@gmail.com>.
Oh, now i finally understand (PHP and JS). And more, just found tests with
usage example (don't now why im just missed it)
Thank you
Waiting for Dmitry
пт, 26 окт. 2018 г. в 18:00, Alexey Kosenchuk <
alexey.kosenchuk@nobitlost.com>:
> thanks, now it's clear.
>
> For NodeJs and PHP clients:
> You cannot use OBJECT_ARRAY constant to specify a field type. This is
> true for all non-primitive types. That's why you get ""type" argument
> has incorrect value" exception.
> Non-primitive types can be specified using special objects. In this
> case - ObjectArrayType.
> All these are described in the API specs.
> So, you need to update your examples:
> PHP: setValueType(new ObjectArrayType())
> NodeJs: setValueType(new ObjectArrayType()) or
> setValueType(new
> ObjectArrayType(ObjectType.PRIMITIVE_TYPE.INTEGER)) (if you want an
> array of integers)
>
> For Python client:
> Dmitry will answer.
>
> Regards,
> -Alexey
>
> 26.10.2018 17:22, Stepan Pilschikov пишет:
> > No
> >
> > Now tried
> > code and output -
> > https://gist.github.com/pilshchikov/4c02057e624baa54326a1e75b4ee3f46
> > And it looks like it works
> >
> > пт, 26 окт. 2018 г. в 17:10, Alexey Kosenchuk
> > <alexey.kosenchuk@nobitlost.com <mailto:alexey.kosenchuk@nobitlost.com
> >>:
> >
> > Hi Stepan,
> >
> > Have you tried the same with Java and .net thin clients as well and
> > everything work fine?
> >
> > Thanks,
> > -Alexey
> >
> > 26.10.2018 16:57, Stepan Pilschikov пишет:
> > > Hi, everyone
> > >
> > > Create new thread to centralize cross compatibility and others
> > common
> > > problems between thin clients
> > >
> > > Tying to use Object array to exchange different data between JS,
> > PHP and
> > > Python thin clients
> > >
> > > JS and PHP simply can't put any type of arrays
> > > Python can put data, but if you take it, data would be completely
> > > different, maybe during this put process data is changed
> > >
> > > Code and output:
> > > PHP -
> > https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
> > > JS -
> > https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
> > > Python -
> > >
> https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
> > >
> > > I'm tried different data types (string, double, float, complex
> > objects,
> > > just random objects, char, byte, Date), still
> > >
> > > How, from my perspective, it should works:
> > > put array of any type and then get this array.
> > > Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new
> > Date()] ->
> > > get [[Date object], [Date object]] (like in java thin client)
> > >
> > > Looks like bug in all clients, what you think?
> > >
> > > I wanted to try Collections, but i think this type have same
> problem
> >
>
Re: Thin clients all in one
Posted by Alexey Kosenchuk <al...@nobitlost.com>.
thanks, now it's clear.
For NodeJs and PHP clients:
You cannot use OBJECT_ARRAY constant to specify a field type. This is
true for all non-primitive types. That's why you get ""type" argument
has incorrect value" exception.
Non-primitive types can be specified using special objects. In this
case - ObjectArrayType.
All these are described in the API specs.
So, you need to update your examples:
PHP: setValueType(new ObjectArrayType())
NodeJs: setValueType(new ObjectArrayType()) or
setValueType(new
ObjectArrayType(ObjectType.PRIMITIVE_TYPE.INTEGER)) (if you want an
array of integers)
For Python client:
Dmitry will answer.
Regards,
-Alexey
26.10.2018 17:22, Stepan Pilschikov пишет:
> No
>
> Now tried
> code and output -
> https://gist.github.com/pilshchikov/4c02057e624baa54326a1e75b4ee3f46
> And it looks like it works
>
> пт, 26 окт. 2018 г. в 17:10, Alexey Kosenchuk
> <alexey.kosenchuk@nobitlost.com <ma...@nobitlost.com>>:
>
> Hi Stepan,
>
> Have you tried the same with Java and .net thin clients as well and
> everything work fine?
>
> Thanks,
> -Alexey
>
> 26.10.2018 16:57, Stepan Pilschikov пишет:
> > Hi, everyone
> >
> > Create new thread to centralize cross compatibility and others
> common
> > problems between thin clients
> >
> > Tying to use Object array to exchange different data between JS,
> PHP and
> > Python thin clients
> >
> > JS and PHP simply can't put any type of arrays
> > Python can put data, but if you take it, data would be completely
> > different, maybe during this put process data is changed
> >
> > Code and output:
> > PHP -
> https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
> > JS -
> https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
> > Python -
> > https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
> >
> > I'm tried different data types (string, double, float, complex
> objects,
> > just random objects, char, byte, Date), still
> >
> > How, from my perspective, it should works:
> > put array of any type and then get this array.
> > Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new
> Date()] ->
> > get [[Date object], [Date object]] (like in java thin client)
> >
> > Looks like bug in all clients, what you think?
> >
> > I wanted to try Collections, but i think this type have same problem
>
Re: Thin clients all in one
Posted by Stepan Pilschikov <pi...@gmail.com>.
No
Now tried
code and output -
https://gist.github.com/pilshchikov/4c02057e624baa54326a1e75b4ee3f46
And it looks like it works
пт, 26 окт. 2018 г. в 17:10, Alexey Kosenchuk <
alexey.kosenchuk@nobitlost.com>:
> Hi Stepan,
>
> Have you tried the same with Java and .net thin clients as well and
> everything work fine?
>
> Thanks,
> -Alexey
>
> 26.10.2018 16:57, Stepan Pilschikov пишет:
> > Hi, everyone
> >
> > Create new thread to centralize cross compatibility and others common
> > problems between thin clients
> >
> > Tying to use Object array to exchange different data between JS, PHP and
> > Python thin clients
> >
> > JS and PHP simply can't put any type of arrays
> > Python can put data, but if you take it, data would be completely
> > different, maybe during this put process data is changed
> >
> > Code and output:
> > PHP -
> https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
> > JS -
> https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
> > Python -
> > https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
> >
> > I'm tried different data types (string, double, float, complex objects,
> > just random objects, char, byte, Date), still
> >
> > How, from my perspective, it should works:
> > put array of any type and then get this array.
> > Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new Date()] ->
> > get [[Date object], [Date object]] (like in java thin client)
> >
> > Looks like bug in all clients, what you think?
> >
> > I wanted to try Collections, but i think this type have same problem
>
Re: Thin clients all in one
Posted by Alexey Kosenchuk <al...@nobitlost.com>.
Hi Stepan,
Have you tried the same with Java and .net thin clients as well and
everything work fine?
Thanks,
-Alexey
26.10.2018 16:57, Stepan Pilschikov пишет:
> Hi, everyone
>
> Create new thread to centralize cross compatibility and others common
> problems between thin clients
>
> Tying to use Object array to exchange different data between JS, PHP and
> Python thin clients
>
> JS and PHP simply can't put any type of arrays
> Python can put data, but if you take it, data would be completely
> different, maybe during this put process data is changed
>
> Code and output:
> PHP - https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
> JS - https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
> Python -
> https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
>
> I'm tried different data types (string, double, float, complex objects,
> just random objects, char, byte, Date), still
>
> How, from my perspective, it should works:
> put array of any type and then get this array.
> Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new Date()] ->
> get [[Date object], [Date object]] (like in java thin client)
>
> Looks like bug in all clients, what you think?
>
> I wanted to try Collections, but i think this type have same problem
Re: Thin clients all in one
Posted by Igor Sapego <is...@apache.org>.
I've created documentation ticket for UUID type - [1].
Now about Timestamp type. Binary protocol specification only specifies
how the data type should be transmitted by the network. The purpose and
main requirements for the protocol is to be compact and fast, it is not
designed to be used by user in raw.
Thus binary protocol does not define nor mandates any type representation
for the specific thin client or its API as it is implied that different
languages
may have different practicies and paradigms.
So binary protocol is not something we should refer to when we are talking
about user API. Protocol should be compact and fast, user API should be
convenient.
[1] - https://issues.apache.org/jira/browse/IGNITE-10721.
Best Regards,
Igor
On Mon, Dec 17, 2018 at 6:18 PM Alexey Kosenchuk <
alexey.kosenchuk@nobitlost.com> wrote:
> Hi Stepan,
>
> 1) About UUID...
>
> UUID binary encoding must be defined in the Binary Protocol
> specification
> (
> https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-uuid-guid-
> )
> So, it's the issue against the spec first.
>
> Regarding the clients..
>
> Python client uses Python UUID class
> (https://docs.python.org/2/library/uuid.html) which, if I understand
> correctly, can operate with different encodings. Dmitry M. can correct me.
>
> NodeJs and PHP do not have standard UUID classes.
> So, the clients return/accept UUID as just a 16-byte array with binary
> encoding inside, exactly as it is read / will be written via the binary
> protocol.
> An application may convert this array to/from a required representation.
>
> 2) About Timestamp...
>
> Python, NodeJs and PHP clients behave strictly according to the Binary
> Protocol specification
> (
> https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-timestamp)
>
> which defines that the second field is "a nanoseconds fraction of a last
> millisecond, which value could be in a range from 0 to 999999."
>
> Regards,
> -Alexey
>
> 14.12.2018 17:41, Stepan Pilschikov пишет:
> > Hello again
> >
> > Starting to check compatibility between thin clients in java/c++ and
> > py/php/nodejs and met some problems
> >
> > 1) Found that UUID mixed in a strange way if we taken it from Java/C++
> > to PY/PHP/JS client and backwards
> > Have issue about it https://issues.apache.org/jira/browse/IGNITE-10691,
> > please look at it
> >
> > 2) Is more like conversation about seamless experience between all thin
> > clients
> > PY/PHP/JS/C++/Java have Timestamp data type which is contain
> > tuple(millis, nanos)
> > Main concern about nanos
> >
> > In PHP/JS/Py Nanos count like "nanoseconds of the last millisecond" but
> > in Java and C++ its like we have millis and same millis but with nanos
> > and its strange for PHP/PY/JS if we take Java how is Base thin client
> >
> > For example in Java we have new Timestamp(111111) and it
> > (fastTime=111000, nanos=111000000)
> > In C++ if we getting this cache we take (millis=111111, nanos=111000000)
> > which is kind a right and same as in java
> > But in py/php/js its look like (millis=111111, nanos=0), its kind a
> > understandable what logic is, but sill different behavior and experience
> > What you thinking about it?
> > For now i can't understand why its done how its done, looks like it
> > should be same but something going wrong
> >
> > пт, 30 нояб. 2018 г. в 01:18, Alexey Kosenchuk
> > <alexey.kosenchuk@nobitlost.com <mailto:alexey.kosenchuk@nobitlost.com
> >>:
> >
> > Hi Stepan,
> >
> > pls check the Ignite cfg you use - see the comments in the jira.
> >
> > Also, the examples executors (including AuthTlsExample) are included
> > into NodeJS test suite in TeamCity which run periodically and
> > successfully passed. Eg. the latest one:
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=2426645&tab=buildResultsDiv&buildTypeId=IgniteTests24Java8_ThinClientNodeJs
> >
> > Regards,
> > -Alexey
> >
> > 28.11.2018 17:08, Stepan Pilschikov пишет:
> > > Hello again
> > >
> > > If NodeJS sources found that example AuthTlsExample.js throwing
> > exception
> > > during execution
> > > Output and grid configuration in
> > > https://issues.apache.org/jira/browse/IGNITE-10447
> > >
> > > Can someone have a look at it?
> > >
> > > вс, 25 нояб. 2018 г. в 19:11, Stepan Pilschikov
> > <pilshchikov.s.n@gmail.com <ma...@gmail.com>>:
> > >
> > >> My bad,
> > >> You right
> > >>
> > >> вс, 25 нояб. 2018 г. в 05:37, Dmitry Melnichuk <
> > >> dmitry.melnichuk@nobitlost.com
> > <ma...@nobitlost.com>>:
> > >>
> > >>> Stepan,
> > >>>
> > >>> AFAIK Map type did always behave correctly on client side, as
> > it does
> > >>> now. This is a corresponding piece of my test suite:
> > >>>
> > >>> ```
> > >>> def test_put_get_map(client):
> > >>>
> > >>> cache = client.get_or_create_cache('test_map_cache')
> > >>>
> > >>> cache.put(
> > >>> 'test_map',
> > >>> (
> > >>> MapObject.HASH_MAP,
> > >>> {
> > >>> (123, IntObject): 'test_data',
> > >>> 456: ((1, [456, 'inner_test_string', 789]),
> > >>> CollectionObject),
> > >>> 'test_key': 32.4,
> > >>> }
> > >>> ),
> > >>> value_hint=MapObject
> > >>> )
> > >>> value = cache.get('test_map')
> > >>> assert value == (MapObject.HASH_MAP, {
> > >>> 123: 'test_data',
> > >>> 456: (1, [456, 'inner_test_string', 789]),
> > >>> 'test_key': 32.4,
> > >>> })
> > >>>
> > >>> ```
> > >>>
> > >>> Or is there another, more specific problem with maps?
> > >>>
> > >>> Dmitry
> > >>>
> > >>> On 11/25/18 3:56 AM, Stepan Pilschikov wrote:
> > >>>> Dmitry,
> > >>>>
> > >>>> Great, checked, now all things woks well
> > >>>> Hope that Igor made review for this PR
> > >>>>
> > >>>> But what about Maps? Looks like different ticket? or it can be
> > done in
> > >>> same
> > >>>> ticket scope?
> > >>>>
> > >>>> пт, 23 нояб. 2018 г. в 23:58, Dmitry Melnichuk <
> > >>>> dmitry.melnichuk@nobitlost.com
> > <ma...@nobitlost.com>>:
> > >>>>
> > >>>>> Stepan,
> > >>>>>
> > >>>>> Sorry, I forgot to update from upstream prior to start
> > working on this
> > >>>>> issue, and thus brought a regression. My bad. Just merged
> > with the
> > >>>>> latest master. Please, check it out again.
> > >>>>>
> > >>>>> Dmitry
> > >>>>>
> > >>>>> On 11/24/18 1:37 AM, Stepan Pilschikov wrote:
> > >>>>>> Dmitry,
> > >>>>>>
> > >>>>>> Iv checked and its actually work
> > >>>>>> But a specially in this branch i found another bug
> > >>>>>> Please look at my last comment:
> > >>>>>>
> > >>>>>
> > >>>
> >
> https://issues.apache.org/jira/browse/IGNITE-10358?focusedCommentId=16697285&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16697285
> > >>>>>>
> > >>>>>> пт, 23 нояб. 2018 г. в 01:21, Dmitry Melnichuk <
> > >>>>>> dmitry.melnichuk@nobitlost.com
> > <ma...@nobitlost.com>>:
> > >>>>>>
> > >>>>>>> Stepan,
> > >>>>>>>
> > >>>>>>> Thank you for your great job in evaluating Python thin
> > client, as
> > >>> well
> > >>>>>>> as other thin clients.
> > >>>>>>>
> > >>>>>>> There was indeed a bug in Python client regarding the
> > handling of
> > >>> type
> > >>>>>>> hints in Collection type. I created a fix and did a PR under
> > >>>>>>> IGNITE-10358 task, but the same PR is also fixes the
> problem in
> > >>>>>>> IGNITE-10230 task.
> > >>>>>>>
> > >>>>>>> As of handling the type mapping in gists you provided, I
> left
> > >>> comments
> > >>>>>>> on both tasks.
> > >>>>>>>
> > >>>>>>> Dmitry
> > >>>>>>>
> > >>>>>>> On 11/21/18 6:37 PM, Stepan Pilschikov wrote:
> > >>>>>>>> Dmitry, Alexey
> > >>>>>>>>
> > >>>>>>>> Thank you for help, this answers help me a lot with
> > understanding
> > >>> how
> > >>>>>>>> clients are work
> > >>>>>>>>
> > >>>>>>>> Not so long time ago i met problem which is have expected
> > behavior,
> > >>> but
> > >>>>>>> its
> > >>>>>>>> may broke some workflows in future for some users
> > >>>>>>>>
> > >>>>>>>> Its all about not specified data types in collections and
> > map's
> > >>>>>>>> All description and examples in
> > >>>>>>>> https://issues.apache.org/jira/browse/IGNITE-10358
> > >>>>>>>>
> > >>>>>>>> Dmitry, can you have a quick look at it and maybe in
> > future somehow
> > >>> fix
> > >>>>>>> it?
> > >>>>>>>>
> > >>>>>>>> пт, 26 окт. 2018 г. в 19:05, Dmitry Melnichuk <
> > >>>>>>>> dmitry.melnichuk@nobitlost.com
> > <ma...@nobitlost.com>>:
> > >>>>>>>>
> > >>>>>>>>> Stepan!
> > >>>>>>>>>
> > >>>>>>>>> TL/DR: what you got with Python client in your gist is an
> > intended
> > >>>>>>>>> behavior.
> > >>>>>>>>>
> > >>>>>>>>> Explanation: As per docs, Object array contains of type
> > ID (which
> > >>> is
> > >>>>>>>>> defaults to -1) and an array of objects.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>
> > >>>>>
> > >>>
> >
> https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-object-array
> > >>>>>>>>>
> > >>>>>>>>> Your gist might be fixed accordingly:
> > >>>>>>>>>
> > >>>>>>>>> ```
> > >>>>>>>>> from pyignite import Client
> > >>>>>>>>> from pyignite.datatypes import *
> > >>>>>>>>>
> > >>>>>>>>> OBJECT_ARRAY_TYPE_ID = -1
> > >>>>>>>>> OBJECT_ARRAY_CONTENTS = [1, 2]
> > >>>>>>>>>
> > >>>>>>>>> client = Client()
> > >>>>>>>>> client.connect('127.0.0.1', 10800)
> > >>>>>>>>> cache = client.get_or_create_cache("PY_OBJECT_ARRAY")
> > >>>>>>>>> cache.put(
> > >>>>>>>>> 1,
> > >>>>>>>>> (OBJECT_ARRAY_TYPE_ID, OBJECT_ARRAY_CONTENTS),
> > >>>>>>>>> key_hint=IntObject,
> > >>>>>>>>> value_hint=ObjectArrayObject,
> > >>>>>>>>> )
> > >>>>>>>>>
> > >>>>>>>>> # Python output: print(cache.get(1))
> > >>>>>>>>> # (-1, [1, 2])
> > >>>>>>>>> ```
> > >>>>>>>>>
> > >>>>>>>>> The situation is similar with Map and Collection, they
> > have types.
> > >>>>> Types
> > >>>>>>>>> and type IDs are mostly useless in Python, but I left
> > them for
> > >>>>>>>>> interoperability reasons. If you think I should kick them
> > out, just
> > >>>>> let
> > >>>>>>>>> me know.
> > >>>>>>>>>
> > >>>>>>>>> The usage of these 3 data types is documented and tested.
> > You can
> > >>>>> refer
> > >>>>>>>>> to the table in “Data types” section:
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>
> > >>>>>
> > >>>
> >
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/datatypes/parsers.html
> > >>>>>>>>>
> > >>>>>>>>> The tests are here:
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>
> > >>>>>
> > >>>
> >
> https://github.com/apache/ignite/blob/master/modules/platforms/python/tests/test_datatypes.py#L116-L124
> > >>>>>>>>>
> > >>>>>>>>> On 10/26/18 11:57 PM, Stepan Pilschikov wrote:
> > >>>>>>>>>> Hi, everyone
> > >>>>>>>>>>
> > >>>>>>>>>> Create new thread to centralize cross compatibility and
> > others
> > >>> common
> > >>>>>>>>>> problems between thin clients
> > >>>>>>>>>>
> > >>>>>>>>>> Tying to use Object array to exchange different data
> > between JS,
> > >>> PHP
> > >>>>>>> and
> > >>>>>>>>>> Python thin clients
> > >>>>>>>>>>
> > >>>>>>>>>> JS and PHP simply can't put any type of arrays
> > >>>>>>>>>> Python can put data, but if you take it, data would be
> > completely
> > >>>>>>>>>> different, maybe during this put process data is changed
> > >>>>>>>>>>
> > >>>>>>>>>> Code and output:
> > >>>>>>>>>> PHP -
> > >>>>>>>>>
> > >>>
> > https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
> > >>>>>>>>>> JS -
> > >>>>>>>>>
> > >>>
> > https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
> > >>>>>>>>>> Python -
> > >>>>>>>>>>
> > >>>
> > https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
> > >>>>>>>>>>
> > >>>>>>>>>> I'm tried different data types (string, double, float,
> > complex
> > >>>>> objects,
> > >>>>>>>>>> just random objects, char, byte, Date), still
> > >>>>>>>>>>
> > >>>>>>>>>> How, from my perspective, it should works:
> > >>>>>>>>>> put array of any type and then get this array.
> > >>>>>>>>>> Example: put [1,2,3] -> get [1,2,3] or put [new Date(),
> new
> > >>> Date()]
> > >>>>> ->
> > >>>>>>>>>> get [[Date object], [Date object]] (like in java thin
> > client)
> > >>>>>>>>>>
> > >>>>>>>>>> Looks like bug in all clients, what you think?
> > >>>>>>>>>>
> > >>>>>>>>>> I wanted to try Collections, but i think this type have
> same
> > >>> problem
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>>
> > >
> >
>
Re: Thin clients all in one
Posted by Alexey Kosenchuk <al...@nobitlost.com>.
Hi Stepan,
1) About UUID...
UUID binary encoding must be defined in the Binary Protocol
specification
(https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-uuid-guid-)
So, it's the issue against the spec first.
Regarding the clients..
Python client uses Python UUID class
(https://docs.python.org/2/library/uuid.html) which, if I understand
correctly, can operate with different encodings. Dmitry M. can correct me.
NodeJs and PHP do not have standard UUID classes.
So, the clients return/accept UUID as just a 16-byte array with binary
encoding inside, exactly as it is read / will be written via the binary
protocol.
An application may convert this array to/from a required representation.
2) About Timestamp...
Python, NodeJs and PHP clients behave strictly according to the Binary
Protocol specification
(https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-timestamp)
which defines that the second field is "a nanoseconds fraction of a last
millisecond, which value could be in a range from 0 to 999999."
Regards,
-Alexey
14.12.2018 17:41, Stepan Pilschikov пишет:
> Hello again
>
> Starting to check compatibility between thin clients in java/c++ and
> py/php/nodejs and met some problems
>
> 1) Found that UUID mixed in a strange way if we taken it from Java/C++
> to PY/PHP/JS client and backwards
> Have issue about it https://issues.apache.org/jira/browse/IGNITE-10691,
> please look at it
>
> 2) Is more like conversation about seamless experience between all thin
> clients
> PY/PHP/JS/C++/Java have Timestamp data type which is contain
> tuple(millis, nanos)
> Main concern about nanos
>
> In PHP/JS/Py Nanos count like "nanoseconds of the last millisecond" but
> in Java and C++ its like we have millis and same millis but with nanos
> and its strange for PHP/PY/JS if we take Java how is Base thin client
>
> For example in Java we have new Timestamp(111111) and it
> (fastTime=111000, nanos=111000000)
> In C++ if we getting this cache we take (millis=111111, nanos=111000000)
> which is kind a right and same as in java
> But in py/php/js its look like (millis=111111, nanos=0), its kind a
> understandable what logic is, but sill different behavior and experience
> What you thinking about it?
> For now i can't understand why its done how its done, looks like it
> should be same but something going wrong
>
> пт, 30 нояб. 2018 г. в 01:18, Alexey Kosenchuk
> <alexey.kosenchuk@nobitlost.com <ma...@nobitlost.com>>:
>
> Hi Stepan,
>
> pls check the Ignite cfg you use - see the comments in the jira.
>
> Also, the examples executors (including AuthTlsExample) are included
> into NodeJS test suite in TeamCity which run periodically and
> successfully passed. Eg. the latest one:
> https://ci.ignite.apache.org/viewLog.html?buildId=2426645&tab=buildResultsDiv&buildTypeId=IgniteTests24Java8_ThinClientNodeJs
>
> Regards,
> -Alexey
>
> 28.11.2018 17:08, Stepan Pilschikov пишет:
> > Hello again
> >
> > If NodeJS sources found that example AuthTlsExample.js throwing
> exception
> > during execution
> > Output and grid configuration in
> > https://issues.apache.org/jira/browse/IGNITE-10447
> >
> > Can someone have a look at it?
> >
> > вс, 25 нояб. 2018 г. в 19:11, Stepan Pilschikov
> <pilshchikov.s.n@gmail.com <ma...@gmail.com>>:
> >
> >> My bad,
> >> You right
> >>
> >> вс, 25 нояб. 2018 г. в 05:37, Dmitry Melnichuk <
> >> dmitry.melnichuk@nobitlost.com
> <ma...@nobitlost.com>>:
> >>
> >>> Stepan,
> >>>
> >>> AFAIK Map type did always behave correctly on client side, as
> it does
> >>> now. This is a corresponding piece of my test suite:
> >>>
> >>> ```
> >>> def test_put_get_map(client):
> >>>
> >>> cache = client.get_or_create_cache('test_map_cache')
> >>>
> >>> cache.put(
> >>> 'test_map',
> >>> (
> >>> MapObject.HASH_MAP,
> >>> {
> >>> (123, IntObject): 'test_data',
> >>> 456: ((1, [456, 'inner_test_string', 789]),
> >>> CollectionObject),
> >>> 'test_key': 32.4,
> >>> }
> >>> ),
> >>> value_hint=MapObject
> >>> )
> >>> value = cache.get('test_map')
> >>> assert value == (MapObject.HASH_MAP, {
> >>> 123: 'test_data',
> >>> 456: (1, [456, 'inner_test_string', 789]),
> >>> 'test_key': 32.4,
> >>> })
> >>>
> >>> ```
> >>>
> >>> Or is there another, more specific problem with maps?
> >>>
> >>> Dmitry
> >>>
> >>> On 11/25/18 3:56 AM, Stepan Pilschikov wrote:
> >>>> Dmitry,
> >>>>
> >>>> Great, checked, now all things woks well
> >>>> Hope that Igor made review for this PR
> >>>>
> >>>> But what about Maps? Looks like different ticket? or it can be
> done in
> >>> same
> >>>> ticket scope?
> >>>>
> >>>> пт, 23 нояб. 2018 г. в 23:58, Dmitry Melnichuk <
> >>>> dmitry.melnichuk@nobitlost.com
> <ma...@nobitlost.com>>:
> >>>>
> >>>>> Stepan,
> >>>>>
> >>>>> Sorry, I forgot to update from upstream prior to start
> working on this
> >>>>> issue, and thus brought a regression. My bad. Just merged
> with the
> >>>>> latest master. Please, check it out again.
> >>>>>
> >>>>> Dmitry
> >>>>>
> >>>>> On 11/24/18 1:37 AM, Stepan Pilschikov wrote:
> >>>>>> Dmitry,
> >>>>>>
> >>>>>> Iv checked and its actually work
> >>>>>> But a specially in this branch i found another bug
> >>>>>> Please look at my last comment:
> >>>>>>
> >>>>>
> >>>
> https://issues.apache.org/jira/browse/IGNITE-10358?focusedCommentId=16697285&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16697285
> >>>>>>
> >>>>>> пт, 23 нояб. 2018 г. в 01:21, Dmitry Melnichuk <
> >>>>>> dmitry.melnichuk@nobitlost.com
> <ma...@nobitlost.com>>:
> >>>>>>
> >>>>>>> Stepan,
> >>>>>>>
> >>>>>>> Thank you for your great job in evaluating Python thin
> client, as
> >>> well
> >>>>>>> as other thin clients.
> >>>>>>>
> >>>>>>> There was indeed a bug in Python client regarding the
> handling of
> >>> type
> >>>>>>> hints in Collection type. I created a fix and did a PR under
> >>>>>>> IGNITE-10358 task, but the same PR is also fixes the problem in
> >>>>>>> IGNITE-10230 task.
> >>>>>>>
> >>>>>>> As of handling the type mapping in gists you provided, I left
> >>> comments
> >>>>>>> on both tasks.
> >>>>>>>
> >>>>>>> Dmitry
> >>>>>>>
> >>>>>>> On 11/21/18 6:37 PM, Stepan Pilschikov wrote:
> >>>>>>>> Dmitry, Alexey
> >>>>>>>>
> >>>>>>>> Thank you for help, this answers help me a lot with
> understanding
> >>> how
> >>>>>>>> clients are work
> >>>>>>>>
> >>>>>>>> Not so long time ago i met problem which is have expected
> behavior,
> >>> but
> >>>>>>> its
> >>>>>>>> may broke some workflows in future for some users
> >>>>>>>>
> >>>>>>>> Its all about not specified data types in collections and
> map's
> >>>>>>>> All description and examples in
> >>>>>>>> https://issues.apache.org/jira/browse/IGNITE-10358
> >>>>>>>>
> >>>>>>>> Dmitry, can you have a quick look at it and maybe in
> future somehow
> >>> fix
> >>>>>>> it?
> >>>>>>>>
> >>>>>>>> пт, 26 окт. 2018 г. в 19:05, Dmitry Melnichuk <
> >>>>>>>> dmitry.melnichuk@nobitlost.com
> <ma...@nobitlost.com>>:
> >>>>>>>>
> >>>>>>>>> Stepan!
> >>>>>>>>>
> >>>>>>>>> TL/DR: what you got with Python client in your gist is an
> intended
> >>>>>>>>> behavior.
> >>>>>>>>>
> >>>>>>>>> Explanation: As per docs, Object array contains of type
> ID (which
> >>> is
> >>>>>>>>> defaults to -1) and an array of objects.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>>
> https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-object-array
> >>>>>>>>>
> >>>>>>>>> Your gist might be fixed accordingly:
> >>>>>>>>>
> >>>>>>>>> ```
> >>>>>>>>> from pyignite import Client
> >>>>>>>>> from pyignite.datatypes import *
> >>>>>>>>>
> >>>>>>>>> OBJECT_ARRAY_TYPE_ID = -1
> >>>>>>>>> OBJECT_ARRAY_CONTENTS = [1, 2]
> >>>>>>>>>
> >>>>>>>>> client = Client()
> >>>>>>>>> client.connect('127.0.0.1', 10800)
> >>>>>>>>> cache = client.get_or_create_cache("PY_OBJECT_ARRAY")
> >>>>>>>>> cache.put(
> >>>>>>>>> 1,
> >>>>>>>>> (OBJECT_ARRAY_TYPE_ID, OBJECT_ARRAY_CONTENTS),
> >>>>>>>>> key_hint=IntObject,
> >>>>>>>>> value_hint=ObjectArrayObject,
> >>>>>>>>> )
> >>>>>>>>>
> >>>>>>>>> # Python output: print(cache.get(1))
> >>>>>>>>> # (-1, [1, 2])
> >>>>>>>>> ```
> >>>>>>>>>
> >>>>>>>>> The situation is similar with Map and Collection, they
> have types.
> >>>>> Types
> >>>>>>>>> and type IDs are mostly useless in Python, but I left
> them for
> >>>>>>>>> interoperability reasons. If you think I should kick them
> out, just
> >>>>> let
> >>>>>>>>> me know.
> >>>>>>>>>
> >>>>>>>>> The usage of these 3 data types is documented and tested.
> You can
> >>>>> refer
> >>>>>>>>> to the table in “Data types” section:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>>
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/datatypes/parsers.html
> >>>>>>>>>
> >>>>>>>>> The tests are here:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>>
> https://github.com/apache/ignite/blob/master/modules/platforms/python/tests/test_datatypes.py#L116-L124
> >>>>>>>>>
> >>>>>>>>> On 10/26/18 11:57 PM, Stepan Pilschikov wrote:
> >>>>>>>>>> Hi, everyone
> >>>>>>>>>>
> >>>>>>>>>> Create new thread to centralize cross compatibility and
> others
> >>> common
> >>>>>>>>>> problems between thin clients
> >>>>>>>>>>
> >>>>>>>>>> Tying to use Object array to exchange different data
> between JS,
> >>> PHP
> >>>>>>> and
> >>>>>>>>>> Python thin clients
> >>>>>>>>>>
> >>>>>>>>>> JS and PHP simply can't put any type of arrays
> >>>>>>>>>> Python can put data, but if you take it, data would be
> completely
> >>>>>>>>>> different, maybe during this put process data is changed
> >>>>>>>>>>
> >>>>>>>>>> Code and output:
> >>>>>>>>>> PHP -
> >>>>>>>>>
> >>>
> https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
> >>>>>>>>>> JS -
> >>>>>>>>>
> >>>
> https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
> >>>>>>>>>> Python -
> >>>>>>>>>>
> >>>
> https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
> >>>>>>>>>>
> >>>>>>>>>> I'm tried different data types (string, double, float,
> complex
> >>>>> objects,
> >>>>>>>>>> just random objects, char, byte, Date), still
> >>>>>>>>>>
> >>>>>>>>>> How, from my perspective, it should works:
> >>>>>>>>>> put array of any type and then get this array.
> >>>>>>>>>> Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new
> >>> Date()]
> >>>>> ->
> >>>>>>>>>> get [[Date object], [Date object]] (like in java thin
> client)
> >>>>>>>>>>
> >>>>>>>>>> Looks like bug in all clients, what you think?
> >>>>>>>>>>
> >>>>>>>>>> I wanted to try Collections, but i think this type have same
> >>> problem
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >
>
Re: Thin clients all in one
Posted by Stepan Pilschikov <pi...@gmail.com>.
Hello again
Starting to check compatibility between thin clients in java/c++ and
py/php/nodejs and met some problems
1) Found that UUID mixed in a strange way if we taken it from Java/C++ to
PY/PHP/JS client and backwards
Have issue about it https://issues.apache.org/jira/browse/IGNITE-10691,
please look at it
2) Is more like conversation about seamless experience between all thin
clients
PY/PHP/JS/C++/Java have Timestamp data type which is contain tuple(millis,
nanos)
Main concern about nanos
In PHP/JS/Py Nanos count like "nanoseconds of the last millisecond" but in
Java and C++ its like we have millis and same millis but with nanos and its
strange for PHP/PY/JS if we take Java how is Base thin client
For example in Java we have new Timestamp(111111) and it (fastTime=111000,
nanos=111000000)
In C++ if we getting this cache we take (millis=111111, nanos=111000000)
which is kind a right and same as in java
But in py/php/js its look like (millis=111111, nanos=0), its kind a
understandable what logic is, but sill different behavior and experience
What you thinking about it?
For now i can't understand why its done how its done, looks like it should
be same but something going wrong
пт, 30 нояб. 2018 г. в 01:18, Alexey Kosenchuk <
alexey.kosenchuk@nobitlost.com>:
> Hi Stepan,
>
> pls check the Ignite cfg you use - see the comments in the jira.
>
> Also, the examples executors (including AuthTlsExample) are included
> into NodeJS test suite in TeamCity which run periodically and
> successfully passed. Eg. the latest one:
>
> https://ci.ignite.apache.org/viewLog.html?buildId=2426645&tab=buildResultsDiv&buildTypeId=IgniteTests24Java8_ThinClientNodeJs
>
> Regards,
> -Alexey
>
> 28.11.2018 17:08, Stepan Pilschikov пишет:
> > Hello again
> >
> > If NodeJS sources found that example AuthTlsExample.js throwing exception
> > during execution
> > Output and grid configuration in
> > https://issues.apache.org/jira/browse/IGNITE-10447
> >
> > Can someone have a look at it?
> >
> > вс, 25 нояб. 2018 г. в 19:11, Stepan Pilschikov <
> pilshchikov.s.n@gmail.com>:
> >
> >> My bad,
> >> You right
> >>
> >> вс, 25 нояб. 2018 г. в 05:37, Dmitry Melnichuk <
> >> dmitry.melnichuk@nobitlost.com>:
> >>
> >>> Stepan,
> >>>
> >>> AFAIK Map type did always behave correctly on client side, as it does
> >>> now. This is a corresponding piece of my test suite:
> >>>
> >>> ```
> >>> def test_put_get_map(client):
> >>>
> >>> cache = client.get_or_create_cache('test_map_cache')
> >>>
> >>> cache.put(
> >>> 'test_map',
> >>> (
> >>> MapObject.HASH_MAP,
> >>> {
> >>> (123, IntObject): 'test_data',
> >>> 456: ((1, [456, 'inner_test_string', 789]),
> >>> CollectionObject),
> >>> 'test_key': 32.4,
> >>> }
> >>> ),
> >>> value_hint=MapObject
> >>> )
> >>> value = cache.get('test_map')
> >>> assert value == (MapObject.HASH_MAP, {
> >>> 123: 'test_data',
> >>> 456: (1, [456, 'inner_test_string', 789]),
> >>> 'test_key': 32.4,
> >>> })
> >>>
> >>> ```
> >>>
> >>> Or is there another, more specific problem with maps?
> >>>
> >>> Dmitry
> >>>
> >>> On 11/25/18 3:56 AM, Stepan Pilschikov wrote:
> >>>> Dmitry,
> >>>>
> >>>> Great, checked, now all things woks well
> >>>> Hope that Igor made review for this PR
> >>>>
> >>>> But what about Maps? Looks like different ticket? or it can be done in
> >>> same
> >>>> ticket scope?
> >>>>
> >>>> пт, 23 нояб. 2018 г. в 23:58, Dmitry Melnichuk <
> >>>> dmitry.melnichuk@nobitlost.com>:
> >>>>
> >>>>> Stepan,
> >>>>>
> >>>>> Sorry, I forgot to update from upstream prior to start working on
> this
> >>>>> issue, and thus brought a regression. My bad. Just merged with the
> >>>>> latest master. Please, check it out again.
> >>>>>
> >>>>> Dmitry
> >>>>>
> >>>>> On 11/24/18 1:37 AM, Stepan Pilschikov wrote:
> >>>>>> Dmitry,
> >>>>>>
> >>>>>> Iv checked and its actually work
> >>>>>> But a specially in this branch i found another bug
> >>>>>> Please look at my last comment:
> >>>>>>
> >>>>>
> >>>
> https://issues.apache.org/jira/browse/IGNITE-10358?focusedCommentId=16697285&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16697285
> >>>>>>
> >>>>>> пт, 23 нояб. 2018 г. в 01:21, Dmitry Melnichuk <
> >>>>>> dmitry.melnichuk@nobitlost.com>:
> >>>>>>
> >>>>>>> Stepan,
> >>>>>>>
> >>>>>>> Thank you for your great job in evaluating Python thin client, as
> >>> well
> >>>>>>> as other thin clients.
> >>>>>>>
> >>>>>>> There was indeed a bug in Python client regarding the handling of
> >>> type
> >>>>>>> hints in Collection type. I created a fix and did a PR under
> >>>>>>> IGNITE-10358 task, but the same PR is also fixes the problem in
> >>>>>>> IGNITE-10230 task.
> >>>>>>>
> >>>>>>> As of handling the type mapping in gists you provided, I left
> >>> comments
> >>>>>>> on both tasks.
> >>>>>>>
> >>>>>>> Dmitry
> >>>>>>>
> >>>>>>> On 11/21/18 6:37 PM, Stepan Pilschikov wrote:
> >>>>>>>> Dmitry, Alexey
> >>>>>>>>
> >>>>>>>> Thank you for help, this answers help me a lot with understanding
> >>> how
> >>>>>>>> clients are work
> >>>>>>>>
> >>>>>>>> Not so long time ago i met problem which is have expected
> behavior,
> >>> but
> >>>>>>> its
> >>>>>>>> may broke some workflows in future for some users
> >>>>>>>>
> >>>>>>>> Its all about not specified data types in collections and map's
> >>>>>>>> All description and examples in
> >>>>>>>> https://issues.apache.org/jira/browse/IGNITE-10358
> >>>>>>>>
> >>>>>>>> Dmitry, can you have a quick look at it and maybe in future
> somehow
> >>> fix
> >>>>>>> it?
> >>>>>>>>
> >>>>>>>> пт, 26 окт. 2018 г. в 19:05, Dmitry Melnichuk <
> >>>>>>>> dmitry.melnichuk@nobitlost.com>:
> >>>>>>>>
> >>>>>>>>> Stepan!
> >>>>>>>>>
> >>>>>>>>> TL/DR: what you got with Python client in your gist is an
> intended
> >>>>>>>>> behavior.
> >>>>>>>>>
> >>>>>>>>> Explanation: As per docs, Object array contains of type ID (which
> >>> is
> >>>>>>>>> defaults to -1) and an array of objects.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>>
> https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-object-array
> >>>>>>>>>
> >>>>>>>>> Your gist might be fixed accordingly:
> >>>>>>>>>
> >>>>>>>>> ```
> >>>>>>>>> from pyignite import Client
> >>>>>>>>> from pyignite.datatypes import *
> >>>>>>>>>
> >>>>>>>>> OBJECT_ARRAY_TYPE_ID = -1
> >>>>>>>>> OBJECT_ARRAY_CONTENTS = [1, 2]
> >>>>>>>>>
> >>>>>>>>> client = Client()
> >>>>>>>>> client.connect('127.0.0.1', 10800)
> >>>>>>>>> cache = client.get_or_create_cache("PY_OBJECT_ARRAY")
> >>>>>>>>> cache.put(
> >>>>>>>>> 1,
> >>>>>>>>> (OBJECT_ARRAY_TYPE_ID, OBJECT_ARRAY_CONTENTS),
> >>>>>>>>> key_hint=IntObject,
> >>>>>>>>> value_hint=ObjectArrayObject,
> >>>>>>>>> )
> >>>>>>>>>
> >>>>>>>>> # Python output: print(cache.get(1))
> >>>>>>>>> # (-1, [1, 2])
> >>>>>>>>> ```
> >>>>>>>>>
> >>>>>>>>> The situation is similar with Map and Collection, they have
> types.
> >>>>> Types
> >>>>>>>>> and type IDs are mostly useless in Python, but I left them for
> >>>>>>>>> interoperability reasons. If you think I should kick them out,
> just
> >>>>> let
> >>>>>>>>> me know.
> >>>>>>>>>
> >>>>>>>>> The usage of these 3 data types is documented and tested. You can
> >>>>> refer
> >>>>>>>>> to the table in “Data types” section:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>>
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/datatypes/parsers.html
> >>>>>>>>>
> >>>>>>>>> The tests are here:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>>
> https://github.com/apache/ignite/blob/master/modules/platforms/python/tests/test_datatypes.py#L116-L124
> >>>>>>>>>
> >>>>>>>>> On 10/26/18 11:57 PM, Stepan Pilschikov wrote:
> >>>>>>>>>> Hi, everyone
> >>>>>>>>>>
> >>>>>>>>>> Create new thread to centralize cross compatibility and others
> >>> common
> >>>>>>>>>> problems between thin clients
> >>>>>>>>>>
> >>>>>>>>>> Tying to use Object array to exchange different data between JS,
> >>> PHP
> >>>>>>> and
> >>>>>>>>>> Python thin clients
> >>>>>>>>>>
> >>>>>>>>>> JS and PHP simply can't put any type of arrays
> >>>>>>>>>> Python can put data, but if you take it, data would be
> completely
> >>>>>>>>>> different, maybe during this put process data is changed
> >>>>>>>>>>
> >>>>>>>>>> Code and output:
> >>>>>>>>>> PHP -
> >>>>>>>>>
> >>> https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
> >>>>>>>>>> JS -
> >>>>>>>>>
> >>> https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
> >>>>>>>>>> Python -
> >>>>>>>>>>
> >>> https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
> >>>>>>>>>>
> >>>>>>>>>> I'm tried different data types (string, double, float, complex
> >>>>> objects,
> >>>>>>>>>> just random objects, char, byte, Date), still
> >>>>>>>>>>
> >>>>>>>>>> How, from my perspective, it should works:
> >>>>>>>>>> put array of any type and then get this array.
> >>>>>>>>>> Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new
> >>> Date()]
> >>>>> ->
> >>>>>>>>>> get [[Date object], [Date object]] (like in java thin client)
> >>>>>>>>>>
> >>>>>>>>>> Looks like bug in all clients, what you think?
> >>>>>>>>>>
> >>>>>>>>>> I wanted to try Collections, but i think this type have same
> >>> problem
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >
>
Re: Thin clients all in one
Posted by Alexey Kosenchuk <al...@nobitlost.com>.
Hi Stepan,
pls check the Ignite cfg you use - see the comments in the jira.
Also, the examples executors (including AuthTlsExample) are included
into NodeJS test suite in TeamCity which run periodically and
successfully passed. Eg. the latest one:
https://ci.ignite.apache.org/viewLog.html?buildId=2426645&tab=buildResultsDiv&buildTypeId=IgniteTests24Java8_ThinClientNodeJs
Regards,
-Alexey
28.11.2018 17:08, Stepan Pilschikov пишет:
> Hello again
>
> If NodeJS sources found that example AuthTlsExample.js throwing exception
> during execution
> Output and grid configuration in
> https://issues.apache.org/jira/browse/IGNITE-10447
>
> Can someone have a look at it?
>
> вс, 25 нояб. 2018 г. в 19:11, Stepan Pilschikov <pi...@gmail.com>:
>
>> My bad,
>> You right
>>
>> вс, 25 нояб. 2018 г. в 05:37, Dmitry Melnichuk <
>> dmitry.melnichuk@nobitlost.com>:
>>
>>> Stepan,
>>>
>>> AFAIK Map type did always behave correctly on client side, as it does
>>> now. This is a corresponding piece of my test suite:
>>>
>>> ```
>>> def test_put_get_map(client):
>>>
>>> cache = client.get_or_create_cache('test_map_cache')
>>>
>>> cache.put(
>>> 'test_map',
>>> (
>>> MapObject.HASH_MAP,
>>> {
>>> (123, IntObject): 'test_data',
>>> 456: ((1, [456, 'inner_test_string', 789]),
>>> CollectionObject),
>>> 'test_key': 32.4,
>>> }
>>> ),
>>> value_hint=MapObject
>>> )
>>> value = cache.get('test_map')
>>> assert value == (MapObject.HASH_MAP, {
>>> 123: 'test_data',
>>> 456: (1, [456, 'inner_test_string', 789]),
>>> 'test_key': 32.4,
>>> })
>>>
>>> ```
>>>
>>> Or is there another, more specific problem with maps?
>>>
>>> Dmitry
>>>
>>> On 11/25/18 3:56 AM, Stepan Pilschikov wrote:
>>>> Dmitry,
>>>>
>>>> Great, checked, now all things woks well
>>>> Hope that Igor made review for this PR
>>>>
>>>> But what about Maps? Looks like different ticket? or it can be done in
>>> same
>>>> ticket scope?
>>>>
>>>> пт, 23 нояб. 2018 г. в 23:58, Dmitry Melnichuk <
>>>> dmitry.melnichuk@nobitlost.com>:
>>>>
>>>>> Stepan,
>>>>>
>>>>> Sorry, I forgot to update from upstream prior to start working on this
>>>>> issue, and thus brought a regression. My bad. Just merged with the
>>>>> latest master. Please, check it out again.
>>>>>
>>>>> Dmitry
>>>>>
>>>>> On 11/24/18 1:37 AM, Stepan Pilschikov wrote:
>>>>>> Dmitry,
>>>>>>
>>>>>> Iv checked and its actually work
>>>>>> But a specially in this branch i found another bug
>>>>>> Please look at my last comment:
>>>>>>
>>>>>
>>> https://issues.apache.org/jira/browse/IGNITE-10358?focusedCommentId=16697285&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16697285
>>>>>>
>>>>>> пт, 23 нояб. 2018 г. в 01:21, Dmitry Melnichuk <
>>>>>> dmitry.melnichuk@nobitlost.com>:
>>>>>>
>>>>>>> Stepan,
>>>>>>>
>>>>>>> Thank you for your great job in evaluating Python thin client, as
>>> well
>>>>>>> as other thin clients.
>>>>>>>
>>>>>>> There was indeed a bug in Python client regarding the handling of
>>> type
>>>>>>> hints in Collection type. I created a fix and did a PR under
>>>>>>> IGNITE-10358 task, but the same PR is also fixes the problem in
>>>>>>> IGNITE-10230 task.
>>>>>>>
>>>>>>> As of handling the type mapping in gists you provided, I left
>>> comments
>>>>>>> on both tasks.
>>>>>>>
>>>>>>> Dmitry
>>>>>>>
>>>>>>> On 11/21/18 6:37 PM, Stepan Pilschikov wrote:
>>>>>>>> Dmitry, Alexey
>>>>>>>>
>>>>>>>> Thank you for help, this answers help me a lot with understanding
>>> how
>>>>>>>> clients are work
>>>>>>>>
>>>>>>>> Not so long time ago i met problem which is have expected behavior,
>>> but
>>>>>>> its
>>>>>>>> may broke some workflows in future for some users
>>>>>>>>
>>>>>>>> Its all about not specified data types in collections and map's
>>>>>>>> All description and examples in
>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-10358
>>>>>>>>
>>>>>>>> Dmitry, can you have a quick look at it and maybe in future somehow
>>> fix
>>>>>>> it?
>>>>>>>>
>>>>>>>> пт, 26 окт. 2018 г. в 19:05, Dmitry Melnichuk <
>>>>>>>> dmitry.melnichuk@nobitlost.com>:
>>>>>>>>
>>>>>>>>> Stepan!
>>>>>>>>>
>>>>>>>>> TL/DR: what you got with Python client in your gist is an intended
>>>>>>>>> behavior.
>>>>>>>>>
>>>>>>>>> Explanation: As per docs, Object array contains of type ID (which
>>> is
>>>>>>>>> defaults to -1) and an array of objects.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>> https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-object-array
>>>>>>>>>
>>>>>>>>> Your gist might be fixed accordingly:
>>>>>>>>>
>>>>>>>>> ```
>>>>>>>>> from pyignite import Client
>>>>>>>>> from pyignite.datatypes import *
>>>>>>>>>
>>>>>>>>> OBJECT_ARRAY_TYPE_ID = -1
>>>>>>>>> OBJECT_ARRAY_CONTENTS = [1, 2]
>>>>>>>>>
>>>>>>>>> client = Client()
>>>>>>>>> client.connect('127.0.0.1', 10800)
>>>>>>>>> cache = client.get_or_create_cache("PY_OBJECT_ARRAY")
>>>>>>>>> cache.put(
>>>>>>>>> 1,
>>>>>>>>> (OBJECT_ARRAY_TYPE_ID, OBJECT_ARRAY_CONTENTS),
>>>>>>>>> key_hint=IntObject,
>>>>>>>>> value_hint=ObjectArrayObject,
>>>>>>>>> )
>>>>>>>>>
>>>>>>>>> # Python output: print(cache.get(1))
>>>>>>>>> # (-1, [1, 2])
>>>>>>>>> ```
>>>>>>>>>
>>>>>>>>> The situation is similar with Map and Collection, they have types.
>>>>> Types
>>>>>>>>> and type IDs are mostly useless in Python, but I left them for
>>>>>>>>> interoperability reasons. If you think I should kick them out, just
>>>>> let
>>>>>>>>> me know.
>>>>>>>>>
>>>>>>>>> The usage of these 3 data types is documented and tested. You can
>>>>> refer
>>>>>>>>> to the table in “Data types” section:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>> https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/datatypes/parsers.html
>>>>>>>>>
>>>>>>>>> The tests are here:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>> https://github.com/apache/ignite/blob/master/modules/platforms/python/tests/test_datatypes.py#L116-L124
>>>>>>>>>
>>>>>>>>> On 10/26/18 11:57 PM, Stepan Pilschikov wrote:
>>>>>>>>>> Hi, everyone
>>>>>>>>>>
>>>>>>>>>> Create new thread to centralize cross compatibility and others
>>> common
>>>>>>>>>> problems between thin clients
>>>>>>>>>>
>>>>>>>>>> Tying to use Object array to exchange different data between JS,
>>> PHP
>>>>>>> and
>>>>>>>>>> Python thin clients
>>>>>>>>>>
>>>>>>>>>> JS and PHP simply can't put any type of arrays
>>>>>>>>>> Python can put data, but if you take it, data would be completely
>>>>>>>>>> different, maybe during this put process data is changed
>>>>>>>>>>
>>>>>>>>>> Code and output:
>>>>>>>>>> PHP -
>>>>>>>>>
>>> https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
>>>>>>>>>> JS -
>>>>>>>>>
>>> https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
>>>>>>>>>> Python -
>>>>>>>>>>
>>> https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
>>>>>>>>>>
>>>>>>>>>> I'm tried different data types (string, double, float, complex
>>>>> objects,
>>>>>>>>>> just random objects, char, byte, Date), still
>>>>>>>>>>
>>>>>>>>>> How, from my perspective, it should works:
>>>>>>>>>> put array of any type and then get this array.
>>>>>>>>>> Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new
>>> Date()]
>>>>> ->
>>>>>>>>>> get [[Date object], [Date object]] (like in java thin client)
>>>>>>>>>>
>>>>>>>>>> Looks like bug in all clients, what you think?
>>>>>>>>>>
>>>>>>>>>> I wanted to try Collections, but i think this type have same
>>> problem
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>
Re: Thin clients all in one
Posted by Stepan Pilschikov <pi...@gmail.com>.
Hello again
If NodeJS sources found that example AuthTlsExample.js throwing exception
during execution
Output and grid configuration in
https://issues.apache.org/jira/browse/IGNITE-10447
Can someone have a look at it?
вс, 25 нояб. 2018 г. в 19:11, Stepan Pilschikov <pi...@gmail.com>:
> My bad,
> You right
>
> вс, 25 нояб. 2018 г. в 05:37, Dmitry Melnichuk <
> dmitry.melnichuk@nobitlost.com>:
>
>> Stepan,
>>
>> AFAIK Map type did always behave correctly on client side, as it does
>> now. This is a corresponding piece of my test suite:
>>
>> ```
>> def test_put_get_map(client):
>>
>> cache = client.get_or_create_cache('test_map_cache')
>>
>> cache.put(
>> 'test_map',
>> (
>> MapObject.HASH_MAP,
>> {
>> (123, IntObject): 'test_data',
>> 456: ((1, [456, 'inner_test_string', 789]),
>> CollectionObject),
>> 'test_key': 32.4,
>> }
>> ),
>> value_hint=MapObject
>> )
>> value = cache.get('test_map')
>> assert value == (MapObject.HASH_MAP, {
>> 123: 'test_data',
>> 456: (1, [456, 'inner_test_string', 789]),
>> 'test_key': 32.4,
>> })
>>
>> ```
>>
>> Or is there another, more specific problem with maps?
>>
>> Dmitry
>>
>> On 11/25/18 3:56 AM, Stepan Pilschikov wrote:
>> > Dmitry,
>> >
>> > Great, checked, now all things woks well
>> > Hope that Igor made review for this PR
>> >
>> > But what about Maps? Looks like different ticket? or it can be done in
>> same
>> > ticket scope?
>> >
>> > пт, 23 нояб. 2018 г. в 23:58, Dmitry Melnichuk <
>> > dmitry.melnichuk@nobitlost.com>:
>> >
>> >> Stepan,
>> >>
>> >> Sorry, I forgot to update from upstream prior to start working on this
>> >> issue, and thus brought a regression. My bad. Just merged with the
>> >> latest master. Please, check it out again.
>> >>
>> >> Dmitry
>> >>
>> >> On 11/24/18 1:37 AM, Stepan Pilschikov wrote:
>> >>> Dmitry,
>> >>>
>> >>> Iv checked and its actually work
>> >>> But a specially in this branch i found another bug
>> >>> Please look at my last comment:
>> >>>
>> >>
>> https://issues.apache.org/jira/browse/IGNITE-10358?focusedCommentId=16697285&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16697285
>> >>>
>> >>> пт, 23 нояб. 2018 г. в 01:21, Dmitry Melnichuk <
>> >>> dmitry.melnichuk@nobitlost.com>:
>> >>>
>> >>>> Stepan,
>> >>>>
>> >>>> Thank you for your great job in evaluating Python thin client, as
>> well
>> >>>> as other thin clients.
>> >>>>
>> >>>> There was indeed a bug in Python client regarding the handling of
>> type
>> >>>> hints in Collection type. I created a fix and did a PR under
>> >>>> IGNITE-10358 task, but the same PR is also fixes the problem in
>> >>>> IGNITE-10230 task.
>> >>>>
>> >>>> As of handling the type mapping in gists you provided, I left
>> comments
>> >>>> on both tasks.
>> >>>>
>> >>>> Dmitry
>> >>>>
>> >>>> On 11/21/18 6:37 PM, Stepan Pilschikov wrote:
>> >>>>> Dmitry, Alexey
>> >>>>>
>> >>>>> Thank you for help, this answers help me a lot with understanding
>> how
>> >>>>> clients are work
>> >>>>>
>> >>>>> Not so long time ago i met problem which is have expected behavior,
>> but
>> >>>> its
>> >>>>> may broke some workflows in future for some users
>> >>>>>
>> >>>>> Its all about not specified data types in collections and map's
>> >>>>> All description and examples in
>> >>>>> https://issues.apache.org/jira/browse/IGNITE-10358
>> >>>>>
>> >>>>> Dmitry, can you have a quick look at it and maybe in future somehow
>> fix
>> >>>> it?
>> >>>>>
>> >>>>> пт, 26 окт. 2018 г. в 19:05, Dmitry Melnichuk <
>> >>>>> dmitry.melnichuk@nobitlost.com>:
>> >>>>>
>> >>>>>> Stepan!
>> >>>>>>
>> >>>>>> TL/DR: what you got with Python client in your gist is an intended
>> >>>>>> behavior.
>> >>>>>>
>> >>>>>> Explanation: As per docs, Object array contains of type ID (which
>> is
>> >>>>>> defaults to -1) and an array of objects.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>
>> >>
>> https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-object-array
>> >>>>>>
>> >>>>>> Your gist might be fixed accordingly:
>> >>>>>>
>> >>>>>> ```
>> >>>>>> from pyignite import Client
>> >>>>>> from pyignite.datatypes import *
>> >>>>>>
>> >>>>>> OBJECT_ARRAY_TYPE_ID = -1
>> >>>>>> OBJECT_ARRAY_CONTENTS = [1, 2]
>> >>>>>>
>> >>>>>> client = Client()
>> >>>>>> client.connect('127.0.0.1', 10800)
>> >>>>>> cache = client.get_or_create_cache("PY_OBJECT_ARRAY")
>> >>>>>> cache.put(
>> >>>>>> 1,
>> >>>>>> (OBJECT_ARRAY_TYPE_ID, OBJECT_ARRAY_CONTENTS),
>> >>>>>> key_hint=IntObject,
>> >>>>>> value_hint=ObjectArrayObject,
>> >>>>>> )
>> >>>>>>
>> >>>>>> # Python output: print(cache.get(1))
>> >>>>>> # (-1, [1, 2])
>> >>>>>> ```
>> >>>>>>
>> >>>>>> The situation is similar with Map and Collection, they have types.
>> >> Types
>> >>>>>> and type IDs are mostly useless in Python, but I left them for
>> >>>>>> interoperability reasons. If you think I should kick them out, just
>> >> let
>> >>>>>> me know.
>> >>>>>>
>> >>>>>> The usage of these 3 data types is documented and tested. You can
>> >> refer
>> >>>>>> to the table in “Data types” section:
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>
>> >>
>> https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/datatypes/parsers.html
>> >>>>>>
>> >>>>>> The tests are here:
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>
>> >>
>> https://github.com/apache/ignite/blob/master/modules/platforms/python/tests/test_datatypes.py#L116-L124
>> >>>>>>
>> >>>>>> On 10/26/18 11:57 PM, Stepan Pilschikov wrote:
>> >>>>>>> Hi, everyone
>> >>>>>>>
>> >>>>>>> Create new thread to centralize cross compatibility and others
>> common
>> >>>>>>> problems between thin clients
>> >>>>>>>
>> >>>>>>> Tying to use Object array to exchange different data between JS,
>> PHP
>> >>>> and
>> >>>>>>> Python thin clients
>> >>>>>>>
>> >>>>>>> JS and PHP simply can't put any type of arrays
>> >>>>>>> Python can put data, but if you take it, data would be completely
>> >>>>>>> different, maybe during this put process data is changed
>> >>>>>>>
>> >>>>>>> Code and output:
>> >>>>>>> PHP -
>> >>>>>>
>> https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
>> >>>>>>> JS -
>> >>>>>>
>> https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
>> >>>>>>> Python -
>> >>>>>>>
>> https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
>> >>>>>>>
>> >>>>>>> I'm tried different data types (string, double, float, complex
>> >> objects,
>> >>>>>>> just random objects, char, byte, Date), still
>> >>>>>>>
>> >>>>>>> How, from my perspective, it should works:
>> >>>>>>> put array of any type and then get this array.
>> >>>>>>> Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new
>> Date()]
>> >> ->
>> >>>>>>> get [[Date object], [Date object]] (like in java thin client)
>> >>>>>>>
>> >>>>>>> Looks like bug in all clients, what you think?
>> >>>>>>>
>> >>>>>>> I wanted to try Collections, but i think this type have same
>> problem
>> >>>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>
>> >>
>> >>
>> >
>>
>>
Re: Thin clients all in one
Posted by Stepan Pilschikov <pi...@gmail.com>.
My bad,
You right
вс, 25 нояб. 2018 г. в 05:37, Dmitry Melnichuk <
dmitry.melnichuk@nobitlost.com>:
> Stepan,
>
> AFAIK Map type did always behave correctly on client side, as it does
> now. This is a corresponding piece of my test suite:
>
> ```
> def test_put_get_map(client):
>
> cache = client.get_or_create_cache('test_map_cache')
>
> cache.put(
> 'test_map',
> (
> MapObject.HASH_MAP,
> {
> (123, IntObject): 'test_data',
> 456: ((1, [456, 'inner_test_string', 789]),
> CollectionObject),
> 'test_key': 32.4,
> }
> ),
> value_hint=MapObject
> )
> value = cache.get('test_map')
> assert value == (MapObject.HASH_MAP, {
> 123: 'test_data',
> 456: (1, [456, 'inner_test_string', 789]),
> 'test_key': 32.4,
> })
>
> ```
>
> Or is there another, more specific problem with maps?
>
> Dmitry
>
> On 11/25/18 3:56 AM, Stepan Pilschikov wrote:
> > Dmitry,
> >
> > Great, checked, now all things woks well
> > Hope that Igor made review for this PR
> >
> > But what about Maps? Looks like different ticket? or it can be done in
> same
> > ticket scope?
> >
> > пт, 23 нояб. 2018 г. в 23:58, Dmitry Melnichuk <
> > dmitry.melnichuk@nobitlost.com>:
> >
> >> Stepan,
> >>
> >> Sorry, I forgot to update from upstream prior to start working on this
> >> issue, and thus brought a regression. My bad. Just merged with the
> >> latest master. Please, check it out again.
> >>
> >> Dmitry
> >>
> >> On 11/24/18 1:37 AM, Stepan Pilschikov wrote:
> >>> Dmitry,
> >>>
> >>> Iv checked and its actually work
> >>> But a specially in this branch i found another bug
> >>> Please look at my last comment:
> >>>
> >>
> https://issues.apache.org/jira/browse/IGNITE-10358?focusedCommentId=16697285&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16697285
> >>>
> >>> пт, 23 нояб. 2018 г. в 01:21, Dmitry Melnichuk <
> >>> dmitry.melnichuk@nobitlost.com>:
> >>>
> >>>> Stepan,
> >>>>
> >>>> Thank you for your great job in evaluating Python thin client, as well
> >>>> as other thin clients.
> >>>>
> >>>> There was indeed a bug in Python client regarding the handling of type
> >>>> hints in Collection type. I created a fix and did a PR under
> >>>> IGNITE-10358 task, but the same PR is also fixes the problem in
> >>>> IGNITE-10230 task.
> >>>>
> >>>> As of handling the type mapping in gists you provided, I left comments
> >>>> on both tasks.
> >>>>
> >>>> Dmitry
> >>>>
> >>>> On 11/21/18 6:37 PM, Stepan Pilschikov wrote:
> >>>>> Dmitry, Alexey
> >>>>>
> >>>>> Thank you for help, this answers help me a lot with understanding how
> >>>>> clients are work
> >>>>>
> >>>>> Not so long time ago i met problem which is have expected behavior,
> but
> >>>> its
> >>>>> may broke some workflows in future for some users
> >>>>>
> >>>>> Its all about not specified data types in collections and map's
> >>>>> All description and examples in
> >>>>> https://issues.apache.org/jira/browse/IGNITE-10358
> >>>>>
> >>>>> Dmitry, can you have a quick look at it and maybe in future somehow
> fix
> >>>> it?
> >>>>>
> >>>>> пт, 26 окт. 2018 г. в 19:05, Dmitry Melnichuk <
> >>>>> dmitry.melnichuk@nobitlost.com>:
> >>>>>
> >>>>>> Stepan!
> >>>>>>
> >>>>>> TL/DR: what you got with Python client in your gist is an intended
> >>>>>> behavior.
> >>>>>>
> >>>>>> Explanation: As per docs, Object array contains of type ID (which is
> >>>>>> defaults to -1) and an array of objects.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>
> https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-object-array
> >>>>>>
> >>>>>> Your gist might be fixed accordingly:
> >>>>>>
> >>>>>> ```
> >>>>>> from pyignite import Client
> >>>>>> from pyignite.datatypes import *
> >>>>>>
> >>>>>> OBJECT_ARRAY_TYPE_ID = -1
> >>>>>> OBJECT_ARRAY_CONTENTS = [1, 2]
> >>>>>>
> >>>>>> client = Client()
> >>>>>> client.connect('127.0.0.1', 10800)
> >>>>>> cache = client.get_or_create_cache("PY_OBJECT_ARRAY")
> >>>>>> cache.put(
> >>>>>> 1,
> >>>>>> (OBJECT_ARRAY_TYPE_ID, OBJECT_ARRAY_CONTENTS),
> >>>>>> key_hint=IntObject,
> >>>>>> value_hint=ObjectArrayObject,
> >>>>>> )
> >>>>>>
> >>>>>> # Python output: print(cache.get(1))
> >>>>>> # (-1, [1, 2])
> >>>>>> ```
> >>>>>>
> >>>>>> The situation is similar with Map and Collection, they have types.
> >> Types
> >>>>>> and type IDs are mostly useless in Python, but I left them for
> >>>>>> interoperability reasons. If you think I should kick them out, just
> >> let
> >>>>>> me know.
> >>>>>>
> >>>>>> The usage of these 3 data types is documented and tested. You can
> >> refer
> >>>>>> to the table in “Data types” section:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/datatypes/parsers.html
> >>>>>>
> >>>>>> The tests are here:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>
> https://github.com/apache/ignite/blob/master/modules/platforms/python/tests/test_datatypes.py#L116-L124
> >>>>>>
> >>>>>> On 10/26/18 11:57 PM, Stepan Pilschikov wrote:
> >>>>>>> Hi, everyone
> >>>>>>>
> >>>>>>> Create new thread to centralize cross compatibility and others
> common
> >>>>>>> problems between thin clients
> >>>>>>>
> >>>>>>> Tying to use Object array to exchange different data between JS,
> PHP
> >>>> and
> >>>>>>> Python thin clients
> >>>>>>>
> >>>>>>> JS and PHP simply can't put any type of arrays
> >>>>>>> Python can put data, but if you take it, data would be completely
> >>>>>>> different, maybe during this put process data is changed
> >>>>>>>
> >>>>>>> Code and output:
> >>>>>>> PHP -
> >>>>>>
> https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
> >>>>>>> JS -
> >>>>>>
> https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
> >>>>>>> Python -
> >>>>>>>
> https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
> >>>>>>>
> >>>>>>> I'm tried different data types (string, double, float, complex
> >> objects,
> >>>>>>> just random objects, char, byte, Date), still
> >>>>>>>
> >>>>>>> How, from my perspective, it should works:
> >>>>>>> put array of any type and then get this array.
> >>>>>>> Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new Date()]
> >> ->
> >>>>>>> get [[Date object], [Date object]] (like in java thin client)
> >>>>>>>
> >>>>>>> Looks like bug in all clients, what you think?
> >>>>>>>
> >>>>>>> I wanted to try Collections, but i think this type have same
> problem
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >>
> >>
> >
>
>
Re: Thin clients all in one
Posted by Dmitry Melnichuk <dm...@nobitlost.com>.
Stepan,
AFAIK Map type did always behave correctly on client side, as it does
now. This is a corresponding piece of my test suite:
```
def test_put_get_map(client):
cache = client.get_or_create_cache('test_map_cache')
cache.put(
'test_map',
(
MapObject.HASH_MAP,
{
(123, IntObject): 'test_data',
456: ((1, [456, 'inner_test_string', 789]),
CollectionObject),
'test_key': 32.4,
}
),
value_hint=MapObject
)
value = cache.get('test_map')
assert value == (MapObject.HASH_MAP, {
123: 'test_data',
456: (1, [456, 'inner_test_string', 789]),
'test_key': 32.4,
})
```
Or is there another, more specific problem with maps?
Dmitry
On 11/25/18 3:56 AM, Stepan Pilschikov wrote:
> Dmitry,
>
> Great, checked, now all things woks well
> Hope that Igor made review for this PR
>
> But what about Maps? Looks like different ticket? or it can be done in same
> ticket scope?
>
> пт, 23 нояб. 2018 г. в 23:58, Dmitry Melnichuk <
> dmitry.melnichuk@nobitlost.com>:
>
>> Stepan,
>>
>> Sorry, I forgot to update from upstream prior to start working on this
>> issue, and thus brought a regression. My bad. Just merged with the
>> latest master. Please, check it out again.
>>
>> Dmitry
>>
>> On 11/24/18 1:37 AM, Stepan Pilschikov wrote:
>>> Dmitry,
>>>
>>> Iv checked and its actually work
>>> But a specially in this branch i found another bug
>>> Please look at my last comment:
>>>
>> https://issues.apache.org/jira/browse/IGNITE-10358?focusedCommentId=16697285&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16697285
>>>
>>> пт, 23 нояб. 2018 г. в 01:21, Dmitry Melnichuk <
>>> dmitry.melnichuk@nobitlost.com>:
>>>
>>>> Stepan,
>>>>
>>>> Thank you for your great job in evaluating Python thin client, as well
>>>> as other thin clients.
>>>>
>>>> There was indeed a bug in Python client regarding the handling of type
>>>> hints in Collection type. I created a fix and did a PR under
>>>> IGNITE-10358 task, but the same PR is also fixes the problem in
>>>> IGNITE-10230 task.
>>>>
>>>> As of handling the type mapping in gists you provided, I left comments
>>>> on both tasks.
>>>>
>>>> Dmitry
>>>>
>>>> On 11/21/18 6:37 PM, Stepan Pilschikov wrote:
>>>>> Dmitry, Alexey
>>>>>
>>>>> Thank you for help, this answers help me a lot with understanding how
>>>>> clients are work
>>>>>
>>>>> Not so long time ago i met problem which is have expected behavior, but
>>>> its
>>>>> may broke some workflows in future for some users
>>>>>
>>>>> Its all about not specified data types in collections and map's
>>>>> All description and examples in
>>>>> https://issues.apache.org/jira/browse/IGNITE-10358
>>>>>
>>>>> Dmitry, can you have a quick look at it and maybe in future somehow fix
>>>> it?
>>>>>
>>>>> пт, 26 окт. 2018 г. в 19:05, Dmitry Melnichuk <
>>>>> dmitry.melnichuk@nobitlost.com>:
>>>>>
>>>>>> Stepan!
>>>>>>
>>>>>> TL/DR: what you got with Python client in your gist is an intended
>>>>>> behavior.
>>>>>>
>>>>>> Explanation: As per docs, Object array contains of type ID (which is
>>>>>> defaults to -1) and an array of objects.
>>>>>>
>>>>>>
>>>>>>
>>>>
>> https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-object-array
>>>>>>
>>>>>> Your gist might be fixed accordingly:
>>>>>>
>>>>>> ```
>>>>>> from pyignite import Client
>>>>>> from pyignite.datatypes import *
>>>>>>
>>>>>> OBJECT_ARRAY_TYPE_ID = -1
>>>>>> OBJECT_ARRAY_CONTENTS = [1, 2]
>>>>>>
>>>>>> client = Client()
>>>>>> client.connect('127.0.0.1', 10800)
>>>>>> cache = client.get_or_create_cache("PY_OBJECT_ARRAY")
>>>>>> cache.put(
>>>>>> 1,
>>>>>> (OBJECT_ARRAY_TYPE_ID, OBJECT_ARRAY_CONTENTS),
>>>>>> key_hint=IntObject,
>>>>>> value_hint=ObjectArrayObject,
>>>>>> )
>>>>>>
>>>>>> # Python output: print(cache.get(1))
>>>>>> # (-1, [1, 2])
>>>>>> ```
>>>>>>
>>>>>> The situation is similar with Map and Collection, they have types.
>> Types
>>>>>> and type IDs are mostly useless in Python, but I left them for
>>>>>> interoperability reasons. If you think I should kick them out, just
>> let
>>>>>> me know.
>>>>>>
>>>>>> The usage of these 3 data types is documented and tested. You can
>> refer
>>>>>> to the table in “Data types” section:
>>>>>>
>>>>>>
>>>>>>
>>>>
>> https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/datatypes/parsers.html
>>>>>>
>>>>>> The tests are here:
>>>>>>
>>>>>>
>>>>>>
>>>>
>> https://github.com/apache/ignite/blob/master/modules/platforms/python/tests/test_datatypes.py#L116-L124
>>>>>>
>>>>>> On 10/26/18 11:57 PM, Stepan Pilschikov wrote:
>>>>>>> Hi, everyone
>>>>>>>
>>>>>>> Create new thread to centralize cross compatibility and others common
>>>>>>> problems between thin clients
>>>>>>>
>>>>>>> Tying to use Object array to exchange different data between JS, PHP
>>>> and
>>>>>>> Python thin clients
>>>>>>>
>>>>>>> JS and PHP simply can't put any type of arrays
>>>>>>> Python can put data, but if you take it, data would be completely
>>>>>>> different, maybe during this put process data is changed
>>>>>>>
>>>>>>> Code and output:
>>>>>>> PHP -
>>>>>> https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
>>>>>>> JS -
>>>>>> https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
>>>>>>> Python -
>>>>>>> https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
>>>>>>>
>>>>>>> I'm tried different data types (string, double, float, complex
>> objects,
>>>>>>> just random objects, char, byte, Date), still
>>>>>>>
>>>>>>> How, from my perspective, it should works:
>>>>>>> put array of any type and then get this array.
>>>>>>> Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new Date()]
>> ->
>>>>>>> get [[Date object], [Date object]] (like in java thin client)
>>>>>>>
>>>>>>> Looks like bug in all clients, what you think?
>>>>>>>
>>>>>>> I wanted to try Collections, but i think this type have same problem
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>
Re: Thin clients all in one
Posted by Stepan Pilschikov <pi...@gmail.com>.
Dmitry,
Great, checked, now all things woks well
Hope that Igor made review for this PR
But what about Maps? Looks like different ticket? or it can be done in same
ticket scope?
пт, 23 нояб. 2018 г. в 23:58, Dmitry Melnichuk <
dmitry.melnichuk@nobitlost.com>:
> Stepan,
>
> Sorry, I forgot to update from upstream prior to start working on this
> issue, and thus brought a regression. My bad. Just merged with the
> latest master. Please, check it out again.
>
> Dmitry
>
> On 11/24/18 1:37 AM, Stepan Pilschikov wrote:
> > Dmitry,
> >
> > Iv checked and its actually work
> > But a specially in this branch i found another bug
> > Please look at my last comment:
> >
> https://issues.apache.org/jira/browse/IGNITE-10358?focusedCommentId=16697285&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16697285
> >
> > пт, 23 нояб. 2018 г. в 01:21, Dmitry Melnichuk <
> > dmitry.melnichuk@nobitlost.com>:
> >
> >> Stepan,
> >>
> >> Thank you for your great job in evaluating Python thin client, as well
> >> as other thin clients.
> >>
> >> There was indeed a bug in Python client regarding the handling of type
> >> hints in Collection type. I created a fix and did a PR under
> >> IGNITE-10358 task, but the same PR is also fixes the problem in
> >> IGNITE-10230 task.
> >>
> >> As of handling the type mapping in gists you provided, I left comments
> >> on both tasks.
> >>
> >> Dmitry
> >>
> >> On 11/21/18 6:37 PM, Stepan Pilschikov wrote:
> >>> Dmitry, Alexey
> >>>
> >>> Thank you for help, this answers help me a lot with understanding how
> >>> clients are work
> >>>
> >>> Not so long time ago i met problem which is have expected behavior, but
> >> its
> >>> may broke some workflows in future for some users
> >>>
> >>> Its all about not specified data types in collections and map's
> >>> All description and examples in
> >>> https://issues.apache.org/jira/browse/IGNITE-10358
> >>>
> >>> Dmitry, can you have a quick look at it and maybe in future somehow fix
> >> it?
> >>>
> >>> пт, 26 окт. 2018 г. в 19:05, Dmitry Melnichuk <
> >>> dmitry.melnichuk@nobitlost.com>:
> >>>
> >>>> Stepan!
> >>>>
> >>>> TL/DR: what you got with Python client in your gist is an intended
> >>>> behavior.
> >>>>
> >>>> Explanation: As per docs, Object array contains of type ID (which is
> >>>> defaults to -1) and an array of objects.
> >>>>
> >>>>
> >>>>
> >>
> https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-object-array
> >>>>
> >>>> Your gist might be fixed accordingly:
> >>>>
> >>>> ```
> >>>> from pyignite import Client
> >>>> from pyignite.datatypes import *
> >>>>
> >>>> OBJECT_ARRAY_TYPE_ID = -1
> >>>> OBJECT_ARRAY_CONTENTS = [1, 2]
> >>>>
> >>>> client = Client()
> >>>> client.connect('127.0.0.1', 10800)
> >>>> cache = client.get_or_create_cache("PY_OBJECT_ARRAY")
> >>>> cache.put(
> >>>> 1,
> >>>> (OBJECT_ARRAY_TYPE_ID, OBJECT_ARRAY_CONTENTS),
> >>>> key_hint=IntObject,
> >>>> value_hint=ObjectArrayObject,
> >>>> )
> >>>>
> >>>> # Python output: print(cache.get(1))
> >>>> # (-1, [1, 2])
> >>>> ```
> >>>>
> >>>> The situation is similar with Map and Collection, they have types.
> Types
> >>>> and type IDs are mostly useless in Python, but I left them for
> >>>> interoperability reasons. If you think I should kick them out, just
> let
> >>>> me know.
> >>>>
> >>>> The usage of these 3 data types is documented and tested. You can
> refer
> >>>> to the table in “Data types” section:
> >>>>
> >>>>
> >>>>
> >>
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/datatypes/parsers.html
> >>>>
> >>>> The tests are here:
> >>>>
> >>>>
> >>>>
> >>
> https://github.com/apache/ignite/blob/master/modules/platforms/python/tests/test_datatypes.py#L116-L124
> >>>>
> >>>> On 10/26/18 11:57 PM, Stepan Pilschikov wrote:
> >>>>> Hi, everyone
> >>>>>
> >>>>> Create new thread to centralize cross compatibility and others common
> >>>>> problems between thin clients
> >>>>>
> >>>>> Tying to use Object array to exchange different data between JS, PHP
> >> and
> >>>>> Python thin clients
> >>>>>
> >>>>> JS and PHP simply can't put any type of arrays
> >>>>> Python can put data, but if you take it, data would be completely
> >>>>> different, maybe during this put process data is changed
> >>>>>
> >>>>> Code and output:
> >>>>> PHP -
> >>>> https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
> >>>>> JS -
> >>>> https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
> >>>>> Python -
> >>>>> https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
> >>>>>
> >>>>> I'm tried different data types (string, double, float, complex
> objects,
> >>>>> just random objects, char, byte, Date), still
> >>>>>
> >>>>> How, from my perspective, it should works:
> >>>>> put array of any type and then get this array.
> >>>>> Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new Date()]
> ->
> >>>>> get [[Date object], [Date object]] (like in java thin client)
> >>>>>
> >>>>> Looks like bug in all clients, what you think?
> >>>>>
> >>>>> I wanted to try Collections, but i think this type have same problem
> >>>>
> >>>
> >>
> >>
> >
>
>
Re: Thin clients all in one
Posted by Dmitry Melnichuk <dm...@nobitlost.com>.
Stepan,
Sorry, I forgot to update from upstream prior to start working on this
issue, and thus brought a regression. My bad. Just merged with the
latest master. Please, check it out again.
Dmitry
On 11/24/18 1:37 AM, Stepan Pilschikov wrote:
> Dmitry,
>
> Iv checked and its actually work
> But a specially in this branch i found another bug
> Please look at my last comment:
> https://issues.apache.org/jira/browse/IGNITE-10358?focusedCommentId=16697285&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16697285
>
> пт, 23 нояб. 2018 г. в 01:21, Dmitry Melnichuk <
> dmitry.melnichuk@nobitlost.com>:
>
>> Stepan,
>>
>> Thank you for your great job in evaluating Python thin client, as well
>> as other thin clients.
>>
>> There was indeed a bug in Python client regarding the handling of type
>> hints in Collection type. I created a fix and did a PR under
>> IGNITE-10358 task, but the same PR is also fixes the problem in
>> IGNITE-10230 task.
>>
>> As of handling the type mapping in gists you provided, I left comments
>> on both tasks.
>>
>> Dmitry
>>
>> On 11/21/18 6:37 PM, Stepan Pilschikov wrote:
>>> Dmitry, Alexey
>>>
>>> Thank you for help, this answers help me a lot with understanding how
>>> clients are work
>>>
>>> Not so long time ago i met problem which is have expected behavior, but
>> its
>>> may broke some workflows in future for some users
>>>
>>> Its all about not specified data types in collections and map's
>>> All description and examples in
>>> https://issues.apache.org/jira/browse/IGNITE-10358
>>>
>>> Dmitry, can you have a quick look at it and maybe in future somehow fix
>> it?
>>>
>>> пт, 26 окт. 2018 г. в 19:05, Dmitry Melnichuk <
>>> dmitry.melnichuk@nobitlost.com>:
>>>
>>>> Stepan!
>>>>
>>>> TL/DR: what you got with Python client in your gist is an intended
>>>> behavior.
>>>>
>>>> Explanation: As per docs, Object array contains of type ID (which is
>>>> defaults to -1) and an array of objects.
>>>>
>>>>
>>>>
>> https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-object-array
>>>>
>>>> Your gist might be fixed accordingly:
>>>>
>>>> ```
>>>> from pyignite import Client
>>>> from pyignite.datatypes import *
>>>>
>>>> OBJECT_ARRAY_TYPE_ID = -1
>>>> OBJECT_ARRAY_CONTENTS = [1, 2]
>>>>
>>>> client = Client()
>>>> client.connect('127.0.0.1', 10800)
>>>> cache = client.get_or_create_cache("PY_OBJECT_ARRAY")
>>>> cache.put(
>>>> 1,
>>>> (OBJECT_ARRAY_TYPE_ID, OBJECT_ARRAY_CONTENTS),
>>>> key_hint=IntObject,
>>>> value_hint=ObjectArrayObject,
>>>> )
>>>>
>>>> # Python output: print(cache.get(1))
>>>> # (-1, [1, 2])
>>>> ```
>>>>
>>>> The situation is similar with Map and Collection, they have types. Types
>>>> and type IDs are mostly useless in Python, but I left them for
>>>> interoperability reasons. If you think I should kick them out, just let
>>>> me know.
>>>>
>>>> The usage of these 3 data types is documented and tested. You can refer
>>>> to the table in “Data types” section:
>>>>
>>>>
>>>>
>> https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/datatypes/parsers.html
>>>>
>>>> The tests are here:
>>>>
>>>>
>>>>
>> https://github.com/apache/ignite/blob/master/modules/platforms/python/tests/test_datatypes.py#L116-L124
>>>>
>>>> On 10/26/18 11:57 PM, Stepan Pilschikov wrote:
>>>>> Hi, everyone
>>>>>
>>>>> Create new thread to centralize cross compatibility and others common
>>>>> problems between thin clients
>>>>>
>>>>> Tying to use Object array to exchange different data between JS, PHP
>> and
>>>>> Python thin clients
>>>>>
>>>>> JS and PHP simply can't put any type of arrays
>>>>> Python can put data, but if you take it, data would be completely
>>>>> different, maybe during this put process data is changed
>>>>>
>>>>> Code and output:
>>>>> PHP -
>>>> https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
>>>>> JS -
>>>> https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
>>>>> Python -
>>>>> https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
>>>>>
>>>>> I'm tried different data types (string, double, float, complex objects,
>>>>> just random objects, char, byte, Date), still
>>>>>
>>>>> How, from my perspective, it should works:
>>>>> put array of any type and then get this array.
>>>>> Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new Date()] ->
>>>>> get [[Date object], [Date object]] (like in java thin client)
>>>>>
>>>>> Looks like bug in all clients, what you think?
>>>>>
>>>>> I wanted to try Collections, but i think this type have same problem
>>>>
>>>
>>
>>
>
Re: Thin clients all in one
Posted by Stepan Pilschikov <pi...@gmail.com>.
Dmitry,
Iv checked and its actually work
But a specially in this branch i found another bug
Please look at my last comment:
https://issues.apache.org/jira/browse/IGNITE-10358?focusedCommentId=16697285&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16697285
пт, 23 нояб. 2018 г. в 01:21, Dmitry Melnichuk <
dmitry.melnichuk@nobitlost.com>:
> Stepan,
>
> Thank you for your great job in evaluating Python thin client, as well
> as other thin clients.
>
> There was indeed a bug in Python client regarding the handling of type
> hints in Collection type. I created a fix and did a PR under
> IGNITE-10358 task, but the same PR is also fixes the problem in
> IGNITE-10230 task.
>
> As of handling the type mapping in gists you provided, I left comments
> on both tasks.
>
> Dmitry
>
> On 11/21/18 6:37 PM, Stepan Pilschikov wrote:
> > Dmitry, Alexey
> >
> > Thank you for help, this answers help me a lot with understanding how
> > clients are work
> >
> > Not so long time ago i met problem which is have expected behavior, but
> its
> > may broke some workflows in future for some users
> >
> > Its all about not specified data types in collections and map's
> > All description and examples in
> > https://issues.apache.org/jira/browse/IGNITE-10358
> >
> > Dmitry, can you have a quick look at it and maybe in future somehow fix
> it?
> >
> > пт, 26 окт. 2018 г. в 19:05, Dmitry Melnichuk <
> > dmitry.melnichuk@nobitlost.com>:
> >
> >> Stepan!
> >>
> >> TL/DR: what you got with Python client in your gist is an intended
> >> behavior.
> >>
> >> Explanation: As per docs, Object array contains of type ID (which is
> >> defaults to -1) and an array of objects.
> >>
> >>
> >>
> https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-object-array
> >>
> >> Your gist might be fixed accordingly:
> >>
> >> ```
> >> from pyignite import Client
> >> from pyignite.datatypes import *
> >>
> >> OBJECT_ARRAY_TYPE_ID = -1
> >> OBJECT_ARRAY_CONTENTS = [1, 2]
> >>
> >> client = Client()
> >> client.connect('127.0.0.1', 10800)
> >> cache = client.get_or_create_cache("PY_OBJECT_ARRAY")
> >> cache.put(
> >> 1,
> >> (OBJECT_ARRAY_TYPE_ID, OBJECT_ARRAY_CONTENTS),
> >> key_hint=IntObject,
> >> value_hint=ObjectArrayObject,
> >> )
> >>
> >> # Python output: print(cache.get(1))
> >> # (-1, [1, 2])
> >> ```
> >>
> >> The situation is similar with Map and Collection, they have types. Types
> >> and type IDs are mostly useless in Python, but I left them for
> >> interoperability reasons. If you think I should kick them out, just let
> >> me know.
> >>
> >> The usage of these 3 data types is documented and tested. You can refer
> >> to the table in “Data types” section:
> >>
> >>
> >>
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/datatypes/parsers.html
> >>
> >> The tests are here:
> >>
> >>
> >>
> https://github.com/apache/ignite/blob/master/modules/platforms/python/tests/test_datatypes.py#L116-L124
> >>
> >> On 10/26/18 11:57 PM, Stepan Pilschikov wrote:
> >>> Hi, everyone
> >>>
> >>> Create new thread to centralize cross compatibility and others common
> >>> problems between thin clients
> >>>
> >>> Tying to use Object array to exchange different data between JS, PHP
> and
> >>> Python thin clients
> >>>
> >>> JS and PHP simply can't put any type of arrays
> >>> Python can put data, but if you take it, data would be completely
> >>> different, maybe during this put process data is changed
> >>>
> >>> Code and output:
> >>> PHP -
> >> https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
> >>> JS -
> >> https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
> >>> Python -
> >>> https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
> >>>
> >>> I'm tried different data types (string, double, float, complex objects,
> >>> just random objects, char, byte, Date), still
> >>>
> >>> How, from my perspective, it should works:
> >>> put array of any type and then get this array.
> >>> Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new Date()] ->
> >>> get [[Date object], [Date object]] (like in java thin client)
> >>>
> >>> Looks like bug in all clients, what you think?
> >>>
> >>> I wanted to try Collections, but i think this type have same problem
> >>
> >
>
>
Re: Thin clients all in one
Posted by Dmitry Melnichuk <dm...@nobitlost.com>.
Stepan,
Thank you for your great job in evaluating Python thin client, as well
as other thin clients.
There was indeed a bug in Python client regarding the handling of type
hints in Collection type. I created a fix and did a PR under
IGNITE-10358 task, but the same PR is also fixes the problem in
IGNITE-10230 task.
As of handling the type mapping in gists you provided, I left comments
on both tasks.
Dmitry
On 11/21/18 6:37 PM, Stepan Pilschikov wrote:
> Dmitry, Alexey
>
> Thank you for help, this answers help me a lot with understanding how
> clients are work
>
> Not so long time ago i met problem which is have expected behavior, but its
> may broke some workflows in future for some users
>
> Its all about not specified data types in collections and map's
> All description and examples in
> https://issues.apache.org/jira/browse/IGNITE-10358
>
> Dmitry, can you have a quick look at it and maybe in future somehow fix it?
>
> пт, 26 окт. 2018 г. в 19:05, Dmitry Melnichuk <
> dmitry.melnichuk@nobitlost.com>:
>
>> Stepan!
>>
>> TL/DR: what you got with Python client in your gist is an intended
>> behavior.
>>
>> Explanation: As per docs, Object array contains of type ID (which is
>> defaults to -1) and an array of objects.
>>
>>
>> https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-object-array
>>
>> Your gist might be fixed accordingly:
>>
>> ```
>> from pyignite import Client
>> from pyignite.datatypes import *
>>
>> OBJECT_ARRAY_TYPE_ID = -1
>> OBJECT_ARRAY_CONTENTS = [1, 2]
>>
>> client = Client()
>> client.connect('127.0.0.1', 10800)
>> cache = client.get_or_create_cache("PY_OBJECT_ARRAY")
>> cache.put(
>> 1,
>> (OBJECT_ARRAY_TYPE_ID, OBJECT_ARRAY_CONTENTS),
>> key_hint=IntObject,
>> value_hint=ObjectArrayObject,
>> )
>>
>> # Python output: print(cache.get(1))
>> # (-1, [1, 2])
>> ```
>>
>> The situation is similar with Map and Collection, they have types. Types
>> and type IDs are mostly useless in Python, but I left them for
>> interoperability reasons. If you think I should kick them out, just let
>> me know.
>>
>> The usage of these 3 data types is documented and tested. You can refer
>> to the table in “Data types” section:
>>
>>
>> https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/datatypes/parsers.html
>>
>> The tests are here:
>>
>>
>> https://github.com/apache/ignite/blob/master/modules/platforms/python/tests/test_datatypes.py#L116-L124
>>
>> On 10/26/18 11:57 PM, Stepan Pilschikov wrote:
>>> Hi, everyone
>>>
>>> Create new thread to centralize cross compatibility and others common
>>> problems between thin clients
>>>
>>> Tying to use Object array to exchange different data between JS, PHP and
>>> Python thin clients
>>>
>>> JS and PHP simply can't put any type of arrays
>>> Python can put data, but if you take it, data would be completely
>>> different, maybe during this put process data is changed
>>>
>>> Code and output:
>>> PHP -
>> https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
>>> JS -
>> https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
>>> Python -
>>> https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
>>>
>>> I'm tried different data types (string, double, float, complex objects,
>>> just random objects, char, byte, Date), still
>>>
>>> How, from my perspective, it should works:
>>> put array of any type and then get this array.
>>> Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new Date()] ->
>>> get [[Date object], [Date object]] (like in java thin client)
>>>
>>> Looks like bug in all clients, what you think?
>>>
>>> I wanted to try Collections, but i think this type have same problem
>>
>
Re: Thin clients all in one
Posted by Stepan Pilschikov <pi...@gmail.com>.
Dmitry, Alexey
Thank you for help, this answers help me a lot with understanding how
clients are work
Not so long time ago i met problem which is have expected behavior, but its
may broke some workflows in future for some users
Its all about not specified data types in collections and map's
All description and examples in
https://issues.apache.org/jira/browse/IGNITE-10358
Dmitry, can you have a quick look at it and maybe in future somehow fix it?
пт, 26 окт. 2018 г. в 19:05, Dmitry Melnichuk <
dmitry.melnichuk@nobitlost.com>:
> Stepan!
>
> TL/DR: what you got with Python client in your gist is an intended
> behavior.
>
> Explanation: As per docs, Object array contains of type ID (which is
> defaults to -1) and an array of objects.
>
>
> https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-object-array
>
> Your gist might be fixed accordingly:
>
> ```
> from pyignite import Client
> from pyignite.datatypes import *
>
> OBJECT_ARRAY_TYPE_ID = -1
> OBJECT_ARRAY_CONTENTS = [1, 2]
>
> client = Client()
> client.connect('127.0.0.1', 10800)
> cache = client.get_or_create_cache("PY_OBJECT_ARRAY")
> cache.put(
> 1,
> (OBJECT_ARRAY_TYPE_ID, OBJECT_ARRAY_CONTENTS),
> key_hint=IntObject,
> value_hint=ObjectArrayObject,
> )
>
> # Python output: print(cache.get(1))
> # (-1, [1, 2])
> ```
>
> The situation is similar with Map and Collection, they have types. Types
> and type IDs are mostly useless in Python, but I left them for
> interoperability reasons. If you think I should kick them out, just let
> me know.
>
> The usage of these 3 data types is documented and tested. You can refer
> to the table in “Data types” section:
>
>
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/datatypes/parsers.html
>
> The tests are here:
>
>
> https://github.com/apache/ignite/blob/master/modules/platforms/python/tests/test_datatypes.py#L116-L124
>
> On 10/26/18 11:57 PM, Stepan Pilschikov wrote:
> > Hi, everyone
> >
> > Create new thread to centralize cross compatibility and others common
> > problems between thin clients
> >
> > Tying to use Object array to exchange different data between JS, PHP and
> > Python thin clients
> >
> > JS and PHP simply can't put any type of arrays
> > Python can put data, but if you take it, data would be completely
> > different, maybe during this put process data is changed
> >
> > Code and output:
> > PHP -
> https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
> > JS -
> https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
> > Python -
> > https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
> >
> > I'm tried different data types (string, double, float, complex objects,
> > just random objects, char, byte, Date), still
> >
> > How, from my perspective, it should works:
> > put array of any type and then get this array.
> > Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new Date()] ->
> > get [[Date object], [Date object]] (like in java thin client)
> >
> > Looks like bug in all clients, what you think?
> >
> > I wanted to try Collections, but i think this type have same problem
>
Re: Thin clients all in one
Posted by Dmitry Melnichuk <dm...@nobitlost.com>.
Stepan!
TL/DR: what you got with Python client in your gist is an intended behavior.
Explanation: As per docs, Object array contains of type ID (which is
defaults to -1) and an array of objects.
https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-object-array
Your gist might be fixed accordingly:
```
from pyignite import Client
from pyignite.datatypes import *
OBJECT_ARRAY_TYPE_ID = -1
OBJECT_ARRAY_CONTENTS = [1, 2]
client = Client()
client.connect('127.0.0.1', 10800)
cache = client.get_or_create_cache("PY_OBJECT_ARRAY")
cache.put(
1,
(OBJECT_ARRAY_TYPE_ID, OBJECT_ARRAY_CONTENTS),
key_hint=IntObject,
value_hint=ObjectArrayObject,
)
# Python output: print(cache.get(1))
# (-1, [1, 2])
```
The situation is similar with Map and Collection, they have types. Types
and type IDs are mostly useless in Python, but I left them for
interoperability reasons. If you think I should kick them out, just let
me know.
The usage of these 3 data types is documented and tested. You can refer
to the table in “Data types” section:
https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/datatypes/parsers.html
The tests are here:
https://github.com/apache/ignite/blob/master/modules/platforms/python/tests/test_datatypes.py#L116-L124
On 10/26/18 11:57 PM, Stepan Pilschikov wrote:
> Hi, everyone
>
> Create new thread to centralize cross compatibility and others common
> problems between thin clients
>
> Tying to use Object array to exchange different data between JS, PHP and
> Python thin clients
>
> JS and PHP simply can't put any type of arrays
> Python can put data, but if you take it, data would be completely
> different, maybe during this put process data is changed
>
> Code and output:
> PHP - https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
> JS - https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
> Python -
> https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
>
> I'm tried different data types (string, double, float, complex objects,
> just random objects, char, byte, Date), still
>
> How, from my perspective, it should works:
> put array of any type and then get this array.
> Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new Date()] ->
> get [[Date object], [Date object]] (like in java thin client)
>
> Looks like bug in all clients, what you think?
>
> I wanted to try Collections, but i think this type have same problem