You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Dmitriy Govorukhin <dm...@gmail.com> on 2017/04/05 10:50:33 UTC
Re: Stable binary key representation
Hi guys, i implemented proxy for IgniteCache in hibernate integration, this
proxy transformate cacheKey to our key wrapper, leaves only required
field. I think we can remove identity resolve, it should not broke
integration with hibernate. Any objections?
On Wed, Mar 29, 2017 at 11:07 PM, Valentin Kulichenko <
valentin.kulichenko@gmail.com> wrote:
> I'm not saying there is no alternative solution. But let's implement it and
> prove that it works first, and remove resolvers only after that.
>
> -Val
>
> On Wed, Mar 29, 2017 at 12:18 PM, Sergi Vladykin <sergi.vladykin@gmail.com
> >
> wrote:
>
> > Guys, nothing is impossible if you know a bit about reflection in Java :)
> >
> > We had a look at the CacheKey class and it is easily replaceable.
> >
> > Sergi
> >
> >
> >
> > 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:
> >
> > > On Wed, Mar 29, 2017 at 11:43 AM, Valentin Kulichenko <
> > > valentin.kulichenko@gmail.com> wrote:
> > >
> > > > "Hibernate key" is the CacheKey class I was referring to. It's
> provided
> > > by
> > > > Hibernate, not by user and not by us. So I'm not sure it's possible
> to
> > > > replace it.
> > > >
> > >
> > > If it is impossible to replace or get rid of the Hibernate key, is this
> > > discussion valid at all?
> > >
> >
>
Re: Stable binary key representation
Posted by Denis Magda <dm...@gridgain.com>.
It's reasonable to listen to Vladimir Ozerov and Alex Paschenko opinion
before any removal.
The identity resolvers were designed by them for specific use cases and, if
I'm not mistaken, one is DML related.
Folks, please join the conversation.
Denis
On Thursday, April 6, 2017, Sergi Vladykin <se...@gmail.com> wrote:
> Val,
>
> I know that you have really vast experience in Ignite deployments and
> probably saw everything that can happen. Did you ever see identity
> resolvers use in real life? I guess no.
>
> Hibernate example is bad here, because if their key is unstable across
> multiple JVMs, it means that it was not designed for distributed caches a
> priori.
>
> Also knowing in advance about stable binary key representation allows us to
> apply additional optimizations, like comparing keys without detaching them
> from offheap memory.
>
> We always will be able to add this stuff back if we see users really need
> it. Let's remove it for 2.0.
>
> Sergi
>
> 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
> valentin.kulichenko@gmail.com <javascript:;>>:
>
> > Alex,
> >
> > To be honest, I don't understand the reasoning behind the removal. I
> think
> > resolvers provide good flexibility for different corner cases and it's a
> > good thing to have them. Note that they can be applied not only to cache
> > keys, but to any binary objects.
> >
> > Hibernate issue is actually a good example of such use case. The fact
> that
> > we found an alternative solution doesn't actually mean anything, because
> > what if this happened not in our module, but in user's application?
> > Unfortunately, we can't predict everything.
> >
> > Error proneness is not a very strong argument either, because in my view
> > these resolvers are as much error prone as BinaryIdMapper, for example.
> >
> > -Val
> >
> > On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com <javascript:;>> wrote:
> >
> > > Denis,
> > >
> > > Can you suggest a use-case where identity resolver is needed (given
> that
> > we
> > > agree that a key must contain only valuable fields)?
> > >
> > > 2017-04-05 22:08 GMT+03:00 Denis Magda <dmagda@apache.org
> <javascript:;>>:
> > >
> > > > Where do you want to remove the identity resolvers from? If it’s
> > related
> > > > to the internals of Hibernate module then it’s fine but if you
> suggest
> > > > removing identity resolvers public interfaces then it might be a
> haste
> > > > decision.
> > > >
> > > > —
> > > > Denis
> > > >
> > > > > On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
> > > alexey.goncharuk@gmail.com <javascript:;>>
> > > > wrote:
> > > > >
> > > > > +1, I see no other reasons to keep it.
> > > > >
> > > > > 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
> sergi.vladykin@gmail.com <javascript:;>
> > >:
> > > > >
> > > > >> +1
> > > > >>
> > > > >> Lets drop them.
> > > > >>
> > > > >> Sergi
> > > > >>
> > > > >> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
> > > > >> dmitriy.govorukhin@gmail.com <javascript:;>>
> > > > >> :
> > > > >>
> > > > >>> Hi guys, i implemented proxy for IgniteCache in hibernate
> > > integration,
> > > > >> this
> > > > >>> proxy transformate cacheKey to our key wrapper, leaves only
> > required
> > > > >>> field. I think we can remove identity resolve, it should not
> broke
> > > > >>> integration with hibernate. Any objections?
> > > > >>>
> > > > >>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin Kulichenko <
> > > > >>> valentin.kulichenko@gmail.com <javascript:;>> wrote:
> > > > >>>
> > > > >>>> I'm not saying there is no alternative solution. But let's
> > implement
> > > > it
> > > > >>> and
> > > > >>>> prove that it works first, and remove resolvers only after that.
> > > > >>>>
> > > > >>>> -Val
> > > > >>>>
> > > > >>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi Vladykin <
> > > > >>> sergi.vladykin@gmail.com <javascript:;>
> > > > >>>>>
> > > > >>>> wrote:
> > > > >>>>
> > > > >>>>> Guys, nothing is impossible if you know a bit about reflection
> in
> > > > >> Java
> > > > >>> :)
> > > > >>>>>
> > > > >>>>> We had a look at the CacheKey class and it is easily
> replaceable.
> > > > >>>>>
> > > > >>>>> Sergi
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
> > > dsetrakyan@apache.org <javascript:;>
> > > > >>> :
> > > > >>>>>
> > > > >>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin Kulichenko <
> > > > >>>>>> valentin.kulichenko@gmail.com <javascript:;>> wrote:
> > > > >>>>>>
> > > > >>>>>>> "Hibernate key" is the CacheKey class I was referring to.
> It's
> > > > >>>> provided
> > > > >>>>>> by
> > > > >>>>>>> Hibernate, not by user and not by us. So I'm not sure it's
> > > > >> possible
> > > > >>>> to
> > > > >>>>>>> replace it.
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>>> If it is impossible to replace or get rid of the Hibernate
> key,
> > is
> > > > >>> this
> > > > >>>>>> discussion valid at all?
> > > > >>>>>>
> > > > >>>>>
> > > > >>>>
> > > > >>>
> > > > >>
> > > >
> > > >
> > >
> >
>
Re: Stable binary key representation
Posted by Denis Magda <dm...@gridgain.com>.
Guys, if this is true
> To my knowledge, Hibernate integration was reworked in 2.0 so that we never
> put Hibernate keys to cache but instead we create our own correct keys
> which guarantee required keys properties.
then who did this? Referring to the discussion in the ticket I’m not sure about that.
—
Denis
> On Apr 19, 2017, at 12:10 AM, Alexey Goncharuk <al...@gmail.com> wrote:
>
> Andrey, do you have a specific unit test which shows the issue? Can you
> check if this is still applicable?
>
> To my knowledge, Hibernate integration was reworked in 2.0 so that we never
> put Hibernate keys to cache but instead we create our own correct keys
> which guarantee required keys properties.
>
> 2017-04-19 4:46 GMT+03:00 Andrey Mashenkov <an...@gmail.com>:
>
>> Denis, I'll rework PR according new solution.
>>
>> Alex G, Sergi, what approach is used for keys comparison in ignite 2.0 ?
>>
>> On Tue, Apr 18, 2017 at 11:11 PM, Denis Magda <dm...@apache.org> wrote:
>>
>>> At all, guys, BinaryIdentityResolvers were discontinued but the ticket
>> [1]
>>> that had triggered the discussion has not been fixed yet.
>>>
>>> It must be fixed in 2.0 otherwise Hibernate integration can be considered
>>> broken in 2.0 because the initial workaround was based on the resolvers.
>>>
>>> Andrey M., will you finalize it. Alex G. and Sergi can suggest
>>> non-resolvers based solution.
>>>
>>> [1] https://issues.apache.org/jira/browse/IGNITE-3429 <
>>> https://issues.apache.org/jira/browse/IGNITE-3429>
>>>
>>> —
>>> Denis
>>>
>>>> On Apr 11, 2017, at 12:06 PM, Denis Magda <dm...@apache.org> wrote:
>>>>
>>>> I don’t see either unless a key’s field is of a float type. However, it
>>> sounds like an artificial use case.
>>>>
>>>> Thanks for the details.
>>>>
>>>> —
>>>> Denis
>>>>
>>>>> On Apr 11, 2017, at 11:50 AM, Dmitriy Setrakyan <
>> dsetrakyan@apache.org>
>>> wrote:
>>>>>
>>>>> Denis, I think it is important that we know which specific field to
>> use
>>> for
>>>>> the affinity resolution, but I don't see any issue in using both,
>>> primary
>>>>> and foreign keys, for hashcode and equality. Do you?
>>>>>
>>>>> D.
>>>>>
>>>>> On Tue, Apr 11, 2017 at 11:46 AM, Igor Sapego <is...@apache.org>
>>> wrote:
>>>>>
>>>>>> Denis,
>>>>>>
>>>>>> The whole binary representation of the object is used now
>>>>>> for hash code generation and equality comparison. So the
>>>>>> answer - all fields are used for this.
>>>>>>
>>>>>> Best Regards,
>>>>>> Igor
>>>>>>
>>>>>> On Mon, Apr 10, 2017 at 9:50 PM, Denis Magda <dm...@apache.org>
>>> wrote:
>>>>>>
>>>>>>> Considering this simple example
>>>>>>>
>>>>>>> INSERT (id, orgId, name, age, address) into Person…
>>>>>>>
>>>>>>> where id and orgId define Person’s affinity key - PersonKey(id,
>> orgId)
>>>>>>>
>>>>>>> How do we know which fields to use for hash code generation and
>>> equality
>>>>>>> comparison? QueryEntity?
>>>>>>>
>>>>>>> No, it’s unclear how to document it properly.
>>>>>>>
>>>>>>> —
>>>>>>> Denis
>>>>>>>
>>>>>>>> On Apr 10, 2017, at 11:14 AM, Vladimir Ozerov <
>> vozerov@gridgain.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> There is no more such resolver. It was removed.
>>>>>>>>
>>>>>>>> On Mon, Apr 10, 2017 at 8:58 PM, Denis Magda <dm...@apache.org>
>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Vovan,
>>>>>>>>>
>>>>>>>>> Before I fix the documentation, what’t the replacement for
>>>>>>>>> BinaryFieldIdentiyResolver we used to define field for hash code
>>>>>>>>> calculation and equality comparison when DML statements are used?
>>>>>>>>> https://apacheignite.readme.io/docs/binary-marshaller#
>>>>>>>>> section-binary-field-identity-resolver <
>> https://apacheignite.readme
>>> .
>>>>>>>>> io/docs/binary-marshaller#section-binary-field-identity-resolver>
>>>>>>>>>
>>>>>>>>> —
>>>>>>>>> Denis
>>>>>>>>>
>>>>>>>>>> On Apr 9, 2017, at 7:39 AM, Vladimir Ozerov <
>> vozerov@gridgain.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Resolvers were essential for DML because we had broken comparison
>>>>>>>>> semantics
>>>>>>>>>> of binary objects. This is not the case now.
>>>>>>>>>>
>>>>>>>>>> Resolver as a whole is normal practice. E.g. it is implemented in
>>>>>> .NET
>>>>>>> on
>>>>>>>>>> core language level and widely used in many cases. Hazelcast has
>> it
>>>>>> as
>>>>>>>>> well
>>>>>>>>>> AFAIK. So it is wrong to think that the whole idea is useless.
>>> Think
>>>>>> of
>>>>>>>>> it
>>>>>>>>>> as a comparator's brother.
>>>>>>>>>>
>>>>>>>>>> The only reason why we need to remove it is missing hash index in
>>> new
>>>>>>>>>> architecture. It makes sense, as it is better to have AI 2.0
>>> without
>>>>>>>>> them,
>>>>>>>>>> than no AI 2.0 :-)
>>>>>>>>>>
>>>>>>>>>> 09 апр. 2017 г. 17:31 пользователь "Sergi Vladykin" <
>>>>>>>>>> sergi.vladykin@gmail.com> написал:
>>>>>>>>>>
>>>>>>>>>>> I guess Resolvers were added to DML just because they already
>>>>>> existed
>>>>>>>>> since
>>>>>>>>>>> 1.9 and we were forced to support them in all the parts of our
>>>>>>> product.
>>>>>>>>>>>
>>>>>>>>>>> We have to stop this practice to add features without clear real
>>>>>> life
>>>>>>>>> use
>>>>>>>>>>> cases for them.
>>>>>>>>>>>
>>>>>>>>>>> Sergi
>>>>>>>>>>>
>>>>>>>>>>> 2017-04-09 17:00 GMT+03:00 Denis Magda <dm...@gridgain.com>:
>>>>>>>>>>>
>>>>>>>>>>>> Sergi, Vovan,
>>>>>>>>>>>>
>>>>>>>>>>>> Sorry for being annoying but I still didn't get an answer on
>>>>>> whether
>>>>>>>>> the
>>>>>>>>>>>> resolvers are the must for DML. The main reason why we made
>> them
>>> up
>>>>>>>>> some
>>>>>>>>>>>> time ago is to support specific DML use cases. However I can't
>>>>>> recall
>>>>>>>>> the
>>>>>>>>>>>> use cases.
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Denis
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <
>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Ok, we need to do 2 things here:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. Drop the resolvers from the source code.
>>>>>>>>>>>>> 2. Write a good page in docs on "What makes a correct cache
>>> key".
>>>>>>>>>>>>>
>>>>>>>>>>>>> Who can do that?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2017-04-07 9:48 GMT+03:00 Sergi Vladykin <
>>>>>> sergi.vladykin@gmail.com
>>>>>>>> :
>>>>>>>>>>>>>
>>>>>>>>>>>>>> It is possible to try adding support of comparison to
>>> Resolvers,
>>>>>>> but
>>>>>>>>>>>> the
>>>>>>>>>>>>>> whole approach looks wrong and for now it is better to get
>> rid
>>> of
>>>>>>> it
>>>>>>>>>>>>> while
>>>>>>>>>>>>>> we have a chance to break compatibility.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
>>>>>>>>>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The discussion should've been started with that :) If
>>> supporting
>>>>>>>>>>>>> resolvers
>>>>>>>>>>>>>>> in new architecture is not possible or means too big effort,
>>>>>> then
>>>>>>>>>>> it's
>>>>>>>>>>>>>>> definitely not worth it.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -Val
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <
>>>>>>>>>>> vozerov@gridgain.com
>>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Dima,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Yes, they may explode some internals of our indexes.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan" <
>>>>>>>>>>>>>>>> dsetrakyan@apache.org> написал:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Guys,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Isn't the main issue here that we cannot use the Identity
>>>>>>>>>>>> Resolvers
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> BTrees in the 2.0 version? If yes, then we have to remove
>>> them
>>>>>>>>>>> no
>>>>>>>>>>>>>>> matter
>>>>>>>>>>>>>>>>> what.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
>>>>>>>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Binary key representation is stable when we always have
>>> equal
>>>>>>>>>>>>>>>> serialized
>>>>>>>>>>>>>>>>>> bytes when the original keys are equal.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Resolver allows you to have some extra info in the Key
>> and
>>>>>>>>>>> equal
>>>>>>>>>>>>>>> Keys
>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>> be serialized into different bytes, which is wrong.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Look at the example what you can do with resolvers:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> We may have some data entry with fields a, b, c. Let's
>> say
>>>>>> the
>>>>>>>>>>>>>>> unique
>>>>>>>>>>>>>>>>> part
>>>>>>>>>>>>>>>>>> here is `a` and it the only fields used in Key equals()
>> and
>>>>>>>>>>>>>>> hashCode().
>>>>>>>>>>>>>>>>>> Still we may have the following layouts:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 1. Ka -> Vbc
>>>>>>>>>>>>>>>>>> 2. Kab -> Vc
>>>>>>>>>>>>>>>>>> 3. Kabc -> Boolean.TRUE
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The only 1 is a correct layout, others are plain wrong
>>>>>>>>>>> variants
>>>>>>>>>>>>> (but
>>>>>>>>>>>>>>>> they
>>>>>>>>>>>>>>>>>> are still possible with Resolvers) because everything
>> that
>>>>>>>>>>> does
>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>> Key unique must be in Value.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> We want to clearly state that if you have something in
>> Key,
>>>>>>>>>>> that
>>>>>>>>>>>>> is
>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>> part of equals(), then the Key is invalid and that stuff
>>> must
>>>>>>>>>>> be
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> Value.
>>>>>>>>>>>>>>>>>> This allows us to rely on binary representation of a Key
>> to
>>>>>> be
>>>>>>>>>>>>>>> stable
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> have some more optimizations and code simplifications
>> with
>>>>>>>>>>>> respect
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>> assumptions.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Even with my vast expirience I would never claim that
>> I've
>>>>>>>>>>>> seen
>>>>>>>>>>>>>>>>>>> "everything" :)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> What do you mean by stable binary key representation and
>>> how
>>>>>>>>>>>>>>>> resolvers
>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>> it unstable?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> -Val
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
>>>>>>>>>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Val,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I know that you have really vast experience in Ignite
>>>>>>>>>>>>>>> deployments
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> probably saw everything that can happen. Did you ever
>> see
>>>>>>>>>>>>>>> identity
>>>>>>>>>>>>>>>>>>>> resolvers use in real life? I guess no.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hibernate example is bad here, because if their key is
>>>>>>>>>>>>> unstable
>>>>>>>>>>>>>>>>> across
>>>>>>>>>>>>>>>>>>>> multiple JVMs, it means that it was not designed for
>>>>>>>>>>>>> distributed
>>>>>>>>>>>>>>>>>> caches a
>>>>>>>>>>>>>>>>>>>> priori.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Also knowing in advance about stable binary key
>>>>>>>>>>>> representation
>>>>>>>>>>>>>>>> allows
>>>>>>>>>>>>>>>>>> us
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> apply additional optimizations, like comparing keys
>>>>>>>>>>> without
>>>>>>>>>>>>>>>> detaching
>>>>>>>>>>>>>>>>>>> them
>>>>>>>>>>>>>>>>>>>> from offheap memory.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> We always will be able to add this stuff back if we see
>>>>>>>>>>>> users
>>>>>>>>>>>>>>>> really
>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>> it. Let's remove it for 2.0.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
>>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Alex,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> To be honest, I don't understand the reasoning behind
>>>>>>>>>>> the
>>>>>>>>>>>>>>>> removal.
>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>> resolvers provide good flexibility for different
>> corner
>>>>>>>>>>>>> cases
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>> good thing to have them. Note that they can be applied
>>>>>>>>>>> not
>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> cache
>>>>>>>>>>>>>>>>>>>>> keys, but to any binary objects.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hibernate issue is actually a good example of such use
>>>>>>>>>>>> case.
>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>> fact
>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>> we found an alternative solution doesn't actually mean
>>>>>>>>>>>>>>> anything,
>>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>> what if this happened not in our module, but in user's
>>>>>>>>>>>>>>>> application?
>>>>>>>>>>>>>>>>>>>>> Unfortunately, we can't predict everything.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Error proneness is not a very strong argument either,
>>>>>>>>>>>>> because
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>> view
>>>>>>>>>>>>>>>>>>>>> these resolvers are as much error prone as
>>>>>>>>>>> BinaryIdMapper,
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> example.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> -Val
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
>>>>>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Denis,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Can you suggest a use-case where identity resolver is
>>>>>>>>>>>>> needed
>>>>>>>>>>>>>>>>> (given
>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>> agree that a key must contain only valuable fields)?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> 2017-04-05 22:08 GMT+03:00 Denis Magda <
>>>>>>>>>>>> dmagda@apache.org
>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Where do you want to remove the identity resolvers
>>>>>>>>>>>> from?
>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>> it’s
>>>>>>>>>>>>>>>>>>>>> related
>>>>>>>>>>>>>>>>>>>>>>> to the internals of Hibernate module then it’s fine
>>>>>>>>>>>> but
>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>> suggest
>>>>>>>>>>>>>>>>>>>>>>> removing identity resolvers public interfaces then
>>>>>>>>>>> it
>>>>>>>>>>>>>>> might
>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>> haste
>>>>>>>>>>>>>>>>>>>>>>> decision.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
>>>>>>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> +1, I see no other reasons to keep it.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
>>>>>>>>>>>>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Lets drop them.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
>>>>>>>>>>>>>>>>>>>>>>>>> dmitriy.govorukhin@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Hi guys, i implemented proxy for IgniteCache in
>>>>>>>>>>>>>>> hibernate
>>>>>>>>>>>>>>>>>>>>>> integration,
>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>> proxy transformate cacheKey to our key wrapper,
>>>>>>>>>>>>> leaves
>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>> required
>>>>>>>>>>>>>>>>>>>>>>>>>> field. I think we can remove identity resolve,
>>>>>>>>>>> it
>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>> broke
>>>>>>>>>>>>>>>>>>>>>>>>>> integration with hibernate. Any objections?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin
>>>>>>>>>>>>> Kulichenko
>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not saying there is no alternative
>>>>>>>>>>> solution.
>>>>>>>>>>>>> But
>>>>>>>>>>>>>>>> let's
>>>>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>> prove that it works first, and remove resolvers
>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> -Val
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi
>>>>>>>>>>> Vladykin
>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Guys, nothing is impossible if you know a bit
>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>> reflection
>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>> Java
>>>>>>>>>>>>>>>>>>>>>>>>>> :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> We had a look at the CacheKey class and it is
>>>>>>>>>>>>> easily
>>>>>>>>>>>>>>>>>>>> replaceable.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
>>>>>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org
>>>>>>>>>>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin
>>>>>>>>>>>>>>> Kulichenko
>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "Hibernate key" is the CacheKey class I was
>>>>>>>>>>>>>>> referring
>>>>>>>>>>>>>>>>> to.
>>>>>>>>>>>>>>>>>>>> It's
>>>>>>>>>>>>>>>>>>>>>>>>>>> provided
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hibernate, not by user and not by us. So I'm
>>>>>>>>>>>> not
>>>>>>>>>>>>>>> sure
>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>> possible
>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> replace it.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If it is impossible to replace or get rid of
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> Hibernate
>>>>>>>>>>>>>>>>>>>> key,
>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> discussion valid at all?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>>>
>>
>>
>> --
>> Best regards,
>> Andrey V. Mashenkov
>>
Re: Stable binary key representation
Posted by Alexey Goncharuk <al...@gmail.com>.
Andrey, do you have a specific unit test which shows the issue? Can you
check if this is still applicable?
To my knowledge, Hibernate integration was reworked in 2.0 so that we never
put Hibernate keys to cache but instead we create our own correct keys
which guarantee required keys properties.
2017-04-19 4:46 GMT+03:00 Andrey Mashenkov <an...@gmail.com>:
> Denis, I'll rework PR according new solution.
>
> Alex G, Sergi, what approach is used for keys comparison in ignite 2.0 ?
>
> On Tue, Apr 18, 2017 at 11:11 PM, Denis Magda <dm...@apache.org> wrote:
>
> > At all, guys, BinaryIdentityResolvers were discontinued but the ticket
> [1]
> > that had triggered the discussion has not been fixed yet.
> >
> > It must be fixed in 2.0 otherwise Hibernate integration can be considered
> > broken in 2.0 because the initial workaround was based on the resolvers.
> >
> > Andrey M., will you finalize it. Alex G. and Sergi can suggest
> > non-resolvers based solution.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-3429 <
> > https://issues.apache.org/jira/browse/IGNITE-3429>
> >
> > —
> > Denis
> >
> > > On Apr 11, 2017, at 12:06 PM, Denis Magda <dm...@apache.org> wrote:
> > >
> > > I don’t see either unless a key’s field is of a float type. However, it
> > sounds like an artificial use case.
> > >
> > > Thanks for the details.
> > >
> > > —
> > > Denis
> > >
> > >> On Apr 11, 2017, at 11:50 AM, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> > wrote:
> > >>
> > >> Denis, I think it is important that we know which specific field to
> use
> > for
> > >> the affinity resolution, but I don't see any issue in using both,
> > primary
> > >> and foreign keys, for hashcode and equality. Do you?
> > >>
> > >> D.
> > >>
> > >> On Tue, Apr 11, 2017 at 11:46 AM, Igor Sapego <is...@apache.org>
> > wrote:
> > >>
> > >>> Denis,
> > >>>
> > >>> The whole binary representation of the object is used now
> > >>> for hash code generation and equality comparison. So the
> > >>> answer - all fields are used for this.
> > >>>
> > >>> Best Regards,
> > >>> Igor
> > >>>
> > >>> On Mon, Apr 10, 2017 at 9:50 PM, Denis Magda <dm...@apache.org>
> > wrote:
> > >>>
> > >>>> Considering this simple example
> > >>>>
> > >>>> INSERT (id, orgId, name, age, address) into Person…
> > >>>>
> > >>>> where id and orgId define Person’s affinity key - PersonKey(id,
> orgId)
> > >>>>
> > >>>> How do we know which fields to use for hash code generation and
> > equality
> > >>>> comparison? QueryEntity?
> > >>>>
> > >>>> No, it’s unclear how to document it properly.
> > >>>>
> > >>>> —
> > >>>> Denis
> > >>>>
> > >>>>> On Apr 10, 2017, at 11:14 AM, Vladimir Ozerov <
> vozerov@gridgain.com>
> > >>>> wrote:
> > >>>>>
> > >>>>> There is no more such resolver. It was removed.
> > >>>>>
> > >>>>> On Mon, Apr 10, 2017 at 8:58 PM, Denis Magda <dm...@apache.org>
> > >>> wrote:
> > >>>>>
> > >>>>>> Vovan,
> > >>>>>>
> > >>>>>> Before I fix the documentation, what’t the replacement for
> > >>>>>> BinaryFieldIdentiyResolver we used to define field for hash code
> > >>>>>> calculation and equality comparison when DML statements are used?
> > >>>>>> https://apacheignite.readme.io/docs/binary-marshaller#
> > >>>>>> section-binary-field-identity-resolver <
> https://apacheignite.readme
> > .
> > >>>>>> io/docs/binary-marshaller#section-binary-field-identity-resolver>
> > >>>>>>
> > >>>>>> —
> > >>>>>> Denis
> > >>>>>>
> > >>>>>>> On Apr 9, 2017, at 7:39 AM, Vladimir Ozerov <
> vozerov@gridgain.com>
> > >>>>>> wrote:
> > >>>>>>>
> > >>>>>>> Resolvers were essential for DML because we had broken comparison
> > >>>>>> semantics
> > >>>>>>> of binary objects. This is not the case now.
> > >>>>>>>
> > >>>>>>> Resolver as a whole is normal practice. E.g. it is implemented in
> > >>> .NET
> > >>>> on
> > >>>>>>> core language level and widely used in many cases. Hazelcast has
> it
> > >>> as
> > >>>>>> well
> > >>>>>>> AFAIK. So it is wrong to think that the whole idea is useless.
> > Think
> > >>> of
> > >>>>>> it
> > >>>>>>> as a comparator's brother.
> > >>>>>>>
> > >>>>>>> The only reason why we need to remove it is missing hash index in
> > new
> > >>>>>>> architecture. It makes sense, as it is better to have AI 2.0
> > without
> > >>>>>> them,
> > >>>>>>> than no AI 2.0 :-)
> > >>>>>>>
> > >>>>>>> 09 апр. 2017 г. 17:31 пользователь "Sergi Vladykin" <
> > >>>>>>> sergi.vladykin@gmail.com> написал:
> > >>>>>>>
> > >>>>>>>> I guess Resolvers were added to DML just because they already
> > >>> existed
> > >>>>>> since
> > >>>>>>>> 1.9 and we were forced to support them in all the parts of our
> > >>>> product.
> > >>>>>>>>
> > >>>>>>>> We have to stop this practice to add features without clear real
> > >>> life
> > >>>>>> use
> > >>>>>>>> cases for them.
> > >>>>>>>>
> > >>>>>>>> Sergi
> > >>>>>>>>
> > >>>>>>>> 2017-04-09 17:00 GMT+03:00 Denis Magda <dm...@gridgain.com>:
> > >>>>>>>>
> > >>>>>>>>> Sergi, Vovan,
> > >>>>>>>>>
> > >>>>>>>>> Sorry for being annoying but I still didn't get an answer on
> > >>> whether
> > >>>>>> the
> > >>>>>>>>> resolvers are the must for DML. The main reason why we made
> them
> > up
> > >>>>>> some
> > >>>>>>>>> time ago is to support specific DML use cases. However I can't
> > >>> recall
> > >>>>>> the
> > >>>>>>>>> use cases.
> > >>>>>>>>>
> > >>>>>>>>> --
> > >>>>>>>>> Denis
> > >>>>>>>>>
> > >>>>>>>>> On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <
> > >>>>>> sergi.vladykin@gmail.com
> > >>>>>>>>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Ok, we need to do 2 things here:
> > >>>>>>>>>>
> > >>>>>>>>>> 1. Drop the resolvers from the source code.
> > >>>>>>>>>> 2. Write a good page in docs on "What makes a correct cache
> > key".
> > >>>>>>>>>>
> > >>>>>>>>>> Who can do that?
> > >>>>>>>>>>
> > >>>>>>>>>> Sergi
> > >>>>>>>>>>
> > >>>>>>>>>> 2017-04-07 9:48 GMT+03:00 Sergi Vladykin <
> > >>> sergi.vladykin@gmail.com
> > >>>>> :
> > >>>>>>>>>>
> > >>>>>>>>>>> It is possible to try adding support of comparison to
> > Resolvers,
> > >>>> but
> > >>>>>>>>> the
> > >>>>>>>>>>> whole approach looks wrong and for now it is better to get
> rid
> > of
> > >>>> it
> > >>>>>>>>>> while
> > >>>>>>>>>>> we have a chance to break compatibility.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Sergi
> > >>>>>>>>>>>
> > >>>>>>>>>>> 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
> > >>>>>>>>>>> valentin.kulichenko@gmail.com>:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> The discussion should've been started with that :) If
> > supporting
> > >>>>>>>>>> resolvers
> > >>>>>>>>>>>> in new architecture is not possible or means too big effort,
> > >>> then
> > >>>>>>>> it's
> > >>>>>>>>>>>> definitely not worth it.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> -Val
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <
> > >>>>>>>> vozerov@gridgain.com
> > >>>>>>>>>>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Dima,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Yes, they may explode some internals of our indexes.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan" <
> > >>>>>>>>>>>>> dsetrakyan@apache.org> написал:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Guys,
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Isn't the main issue here that we cannot use the Identity
> > >>>>>>>>> Resolvers
> > >>>>>>>>>> in
> > >>>>>>>>>>>>>> BTrees in the 2.0 version? If yes, then we have to remove
> > them
> > >>>>>>>> no
> > >>>>>>>>>>>> matter
> > >>>>>>>>>>>>>> what.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> D.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
> > >>>>>>>>>>>> sergi.vladykin@gmail.com
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Binary key representation is stable when we always have
> > equal
> > >>>>>>>>>>>>> serialized
> > >>>>>>>>>>>>>>> bytes when the original keys are equal.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Resolver allows you to have some extra info in the Key
> and
> > >>>>>>>> equal
> > >>>>>>>>>>>> Keys
> > >>>>>>>>>>>>>> will
> > >>>>>>>>>>>>>>> be serialized into different bytes, which is wrong.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Look at the example what you can do with resolvers:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> We may have some data entry with fields a, b, c. Let's
> say
> > >>> the
> > >>>>>>>>>>>> unique
> > >>>>>>>>>>>>>> part
> > >>>>>>>>>>>>>>> here is `a` and it the only fields used in Key equals()
> and
> > >>>>>>>>>>>> hashCode().
> > >>>>>>>>>>>>>>> Still we may have the following layouts:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> 1. Ka -> Vbc
> > >>>>>>>>>>>>>>> 2. Kab -> Vc
> > >>>>>>>>>>>>>>> 3. Kabc -> Boolean.TRUE
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> The only 1 is a correct layout, others are plain wrong
> > >>>>>>>> variants
> > >>>>>>>>>> (but
> > >>>>>>>>>>>>> they
> > >>>>>>>>>>>>>>> are still possible with Resolvers) because everything
> that
> > >>>>>>>> does
> > >>>>>>>>>> not
> > >>>>>>>>>>>>> make
> > >>>>>>>>>>>>>>> Key unique must be in Value.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> We want to clearly state that if you have something in
> Key,
> > >>>>>>>> that
> > >>>>>>>>>> is
> > >>>>>>>>>>>> not
> > >>>>>>>>>>>>>>> part of equals(), then the Key is invalid and that stuff
> > must
> > >>>>>>>> be
> > >>>>>>>>>> in
> > >>>>>>>>>>>>>> Value.
> > >>>>>>>>>>>>>>> This allows us to rely on binary representation of a Key
> to
> > >>> be
> > >>>>>>>>>>>> stable
> > >>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>> have some more optimizations and code simplifications
> with
> > >>>>>>>>> respect
> > >>>>>>>>>>>> to
> > >>>>>>>>>>>>>> these
> > >>>>>>>>>>>>>>> assumptions.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Sergi
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
> > >>>>>>>>>>>>>>> valentin.kulichenko@gmail.com>:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Even with my vast expirience I would never claim that
> I've
> > >>>>>>>>> seen
> > >>>>>>>>>>>>>>>> "everything" :)
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> What do you mean by stable binary key representation and
> > how
> > >>>>>>>>>>>>> resolvers
> > >>>>>>>>>>>>>>> make
> > >>>>>>>>>>>>>>>> it unstable?
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> -Val
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
> > >>>>>>>>>>>>>> sergi.vladykin@gmail.com
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Val,
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> I know that you have really vast experience in Ignite
> > >>>>>>>>>>>> deployments
> > >>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>> probably saw everything that can happen. Did you ever
> see
> > >>>>>>>>>>>> identity
> > >>>>>>>>>>>>>>>>> resolvers use in real life? I guess no.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Hibernate example is bad here, because if their key is
> > >>>>>>>>>> unstable
> > >>>>>>>>>>>>>> across
> > >>>>>>>>>>>>>>>>> multiple JVMs, it means that it was not designed for
> > >>>>>>>>>> distributed
> > >>>>>>>>>>>>>>> caches a
> > >>>>>>>>>>>>>>>>> priori.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Also knowing in advance about stable binary key
> > >>>>>>>>> representation
> > >>>>>>>>>>>>> allows
> > >>>>>>>>>>>>>>> us
> > >>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>> apply additional optimizations, like comparing keys
> > >>>>>>>> without
> > >>>>>>>>>>>>> detaching
> > >>>>>>>>>>>>>>>> them
> > >>>>>>>>>>>>>>>>> from offheap memory.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> We always will be able to add this stuff back if we see
> > >>>>>>>>> users
> > >>>>>>>>>>>>> really
> > >>>>>>>>>>>>>>> need
> > >>>>>>>>>>>>>>>>> it. Let's remove it for 2.0.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Sergi
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
> > >>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com>:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Alex,
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> To be honest, I don't understand the reasoning behind
> > >>>>>>>> the
> > >>>>>>>>>>>>> removal.
> > >>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>>> think
> > >>>>>>>>>>>>>>>>>> resolvers provide good flexibility for different
> corner
> > >>>>>>>>>> cases
> > >>>>>>>>>>>> and
> > >>>>>>>>>>>>>>> it's
> > >>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>> good thing to have them. Note that they can be applied
> > >>>>>>>> not
> > >>>>>>>>>>>> only
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>> cache
> > >>>>>>>>>>>>>>>>>> keys, but to any binary objects.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Hibernate issue is actually a good example of such use
> > >>>>>>>>> case.
> > >>>>>>>>>>>> The
> > >>>>>>>>>>>>>> fact
> > >>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>> we found an alternative solution doesn't actually mean
> > >>>>>>>>>>>> anything,
> > >>>>>>>>>>>>>>>> because
> > >>>>>>>>>>>>>>>>>> what if this happened not in our module, but in user's
> > >>>>>>>>>>>>> application?
> > >>>>>>>>>>>>>>>>>> Unfortunately, we can't predict everything.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Error proneness is not a very strong argument either,
> > >>>>>>>>>> because
> > >>>>>>>>>>>> in
> > >>>>>>>>>>>>> my
> > >>>>>>>>>>>>>>>> view
> > >>>>>>>>>>>>>>>>>> these resolvers are as much error prone as
> > >>>>>>>> BinaryIdMapper,
> > >>>>>>>>>> for
> > >>>>>>>>>>>>>>> example.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> -Val
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
> > >>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Denis,
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Can you suggest a use-case where identity resolver is
> > >>>>>>>>>> needed
> > >>>>>>>>>>>>>> (given
> > >>>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>> agree that a key must contain only valuable fields)?
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> 2017-04-05 22:08 GMT+03:00 Denis Magda <
> > >>>>>>>>> dmagda@apache.org
> > >>>>>>>>>>> :
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Where do you want to remove the identity resolvers
> > >>>>>>>>> from?
> > >>>>>>>>>>>> If
> > >>>>>>>>>>>>>> it’s
> > >>>>>>>>>>>>>>>>>> related
> > >>>>>>>>>>>>>>>>>>>> to the internals of Hibernate module then it’s fine
> > >>>>>>>>> but
> > >>>>>>>>>> if
> > >>>>>>>>>>>>> you
> > >>>>>>>>>>>>>>>>> suggest
> > >>>>>>>>>>>>>>>>>>>> removing identity resolvers public interfaces then
> > >>>>>>>> it
> > >>>>>>>>>>>> might
> > >>>>>>>>>>>>> be
> > >>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>> haste
> > >>>>>>>>>>>>>>>>>>>> decision.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> —
> > >>>>>>>>>>>>>>>>>>>> Denis
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
> > >>>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com>
> > >>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> +1, I see no other reasons to keep it.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
> > >>>>>>>>>>>>>>>>> sergi.vladykin@gmail.com
> > >>>>>>>>>>>>>>>>>>> :
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> +1
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Lets drop them.
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Sergi
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
> > >>>>>>>>>>>>>>>>>>>>>> dmitriy.govorukhin@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>> :
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> Hi guys, i implemented proxy for IgniteCache in
> > >>>>>>>>>>>> hibernate
> > >>>>>>>>>>>>>>>>>>> integration,
> > >>>>>>>>>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>>>>> proxy transformate cacheKey to our key wrapper,
> > >>>>>>>>>> leaves
> > >>>>>>>>>>>>> only
> > >>>>>>>>>>>>>>>>>> required
> > >>>>>>>>>>>>>>>>>>>>>>> field. I think we can remove identity resolve,
> > >>>>>>>> it
> > >>>>>>>>>>>> should
> > >>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>>>> broke
> > >>>>>>>>>>>>>>>>>>>>>>> integration with hibernate. Any objections?
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin
> > >>>>>>>>>> Kulichenko
> > >>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> I'm not saying there is no alternative
> > >>>>>>>> solution.
> > >>>>>>>>>> But
> > >>>>>>>>>>>>> let's
> > >>>>>>>>>>>>>>>>>> implement
> > >>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>> prove that it works first, and remove resolvers
> > >>>>>>>>>> only
> > >>>>>>>>>>>>> after
> > >>>>>>>>>>>>>>>> that.
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> -Val
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi
> > >>>>>>>> Vladykin
> > >>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>> sergi.vladykin@gmail.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> Guys, nothing is impossible if you know a bit
> > >>>>>>>>>> about
> > >>>>>>>>>>>>>>>> reflection
> > >>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>> Java
> > >>>>>>>>>>>>>>>>>>>>>>> :)
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> We had a look at the CacheKey class and it is
> > >>>>>>>>>> easily
> > >>>>>>>>>>>>>>>>> replaceable.
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> Sergi
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
> > >>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org
> > >>>>>>>>>>>>>>>>>>>>>>> :
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin
> > >>>>>>>>>>>> Kulichenko
> > >>>>>>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> "Hibernate key" is the CacheKey class I was
> > >>>>>>>>>>>> referring
> > >>>>>>>>>>>>>> to.
> > >>>>>>>>>>>>>>>>> It's
> > >>>>>>>>>>>>>>>>>>>>>>>> provided
> > >>>>>>>>>>>>>>>>>>>>>>>>>> by
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Hibernate, not by user and not by us. So I'm
> > >>>>>>>>> not
> > >>>>>>>>>>>> sure
> > >>>>>>>>>>>>>>> it's
> > >>>>>>>>>>>>>>>>>>>>>> possible
> > >>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> replace it.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> If it is impossible to replace or get rid of
> > >>>>>>>>> the
> > >>>>>>>>>>>>>> Hibernate
> > >>>>>>>>>>>>>>>>> key,
> > >>>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>>>>>>>> discussion valid at all?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>
> > >>>>
> > >>>
> > >
> >
> >
>
>
> --
> Best regards,
> Andrey V. Mashenkov
>
Re: Stable binary key representation
Posted by Andrey Mashenkov <an...@gmail.com>.
Denis, I'll rework PR according new solution.
Alex G, Sergi, what approach is used for keys comparison in ignite 2.0 ?
On Tue, Apr 18, 2017 at 11:11 PM, Denis Magda <dm...@apache.org> wrote:
> At all, guys, BinaryIdentityResolvers were discontinued but the ticket [1]
> that had triggered the discussion has not been fixed yet.
>
> It must be fixed in 2.0 otherwise Hibernate integration can be considered
> broken in 2.0 because the initial workaround was based on the resolvers.
>
> Andrey M., will you finalize it. Alex G. and Sergi can suggest
> non-resolvers based solution.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-3429 <
> https://issues.apache.org/jira/browse/IGNITE-3429>
>
> —
> Denis
>
> > On Apr 11, 2017, at 12:06 PM, Denis Magda <dm...@apache.org> wrote:
> >
> > I don’t see either unless a key’s field is of a float type. However, it
> sounds like an artificial use case.
> >
> > Thanks for the details.
> >
> > —
> > Denis
> >
> >> On Apr 11, 2017, at 11:50 AM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
> >>
> >> Denis, I think it is important that we know which specific field to use
> for
> >> the affinity resolution, but I don't see any issue in using both,
> primary
> >> and foreign keys, for hashcode and equality. Do you?
> >>
> >> D.
> >>
> >> On Tue, Apr 11, 2017 at 11:46 AM, Igor Sapego <is...@apache.org>
> wrote:
> >>
> >>> Denis,
> >>>
> >>> The whole binary representation of the object is used now
> >>> for hash code generation and equality comparison. So the
> >>> answer - all fields are used for this.
> >>>
> >>> Best Regards,
> >>> Igor
> >>>
> >>> On Mon, Apr 10, 2017 at 9:50 PM, Denis Magda <dm...@apache.org>
> wrote:
> >>>
> >>>> Considering this simple example
> >>>>
> >>>> INSERT (id, orgId, name, age, address) into Person…
> >>>>
> >>>> where id and orgId define Person’s affinity key - PersonKey(id, orgId)
> >>>>
> >>>> How do we know which fields to use for hash code generation and
> equality
> >>>> comparison? QueryEntity?
> >>>>
> >>>> No, it’s unclear how to document it properly.
> >>>>
> >>>> —
> >>>> Denis
> >>>>
> >>>>> On Apr 10, 2017, at 11:14 AM, Vladimir Ozerov <vo...@gridgain.com>
> >>>> wrote:
> >>>>>
> >>>>> There is no more such resolver. It was removed.
> >>>>>
> >>>>> On Mon, Apr 10, 2017 at 8:58 PM, Denis Magda <dm...@apache.org>
> >>> wrote:
> >>>>>
> >>>>>> Vovan,
> >>>>>>
> >>>>>> Before I fix the documentation, what’t the replacement for
> >>>>>> BinaryFieldIdentiyResolver we used to define field for hash code
> >>>>>> calculation and equality comparison when DML statements are used?
> >>>>>> https://apacheignite.readme.io/docs/binary-marshaller#
> >>>>>> section-binary-field-identity-resolver <https://apacheignite.readme
> .
> >>>>>> io/docs/binary-marshaller#section-binary-field-identity-resolver>
> >>>>>>
> >>>>>> —
> >>>>>> Denis
> >>>>>>
> >>>>>>> On Apr 9, 2017, at 7:39 AM, Vladimir Ozerov <vo...@gridgain.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> Resolvers were essential for DML because we had broken comparison
> >>>>>> semantics
> >>>>>>> of binary objects. This is not the case now.
> >>>>>>>
> >>>>>>> Resolver as a whole is normal practice. E.g. it is implemented in
> >>> .NET
> >>>> on
> >>>>>>> core language level and widely used in many cases. Hazelcast has it
> >>> as
> >>>>>> well
> >>>>>>> AFAIK. So it is wrong to think that the whole idea is useless.
> Think
> >>> of
> >>>>>> it
> >>>>>>> as a comparator's brother.
> >>>>>>>
> >>>>>>> The only reason why we need to remove it is missing hash index in
> new
> >>>>>>> architecture. It makes sense, as it is better to have AI 2.0
> without
> >>>>>> them,
> >>>>>>> than no AI 2.0 :-)
> >>>>>>>
> >>>>>>> 09 апр. 2017 г. 17:31 пользователь "Sergi Vladykin" <
> >>>>>>> sergi.vladykin@gmail.com> написал:
> >>>>>>>
> >>>>>>>> I guess Resolvers were added to DML just because they already
> >>> existed
> >>>>>> since
> >>>>>>>> 1.9 and we were forced to support them in all the parts of our
> >>>> product.
> >>>>>>>>
> >>>>>>>> We have to stop this practice to add features without clear real
> >>> life
> >>>>>> use
> >>>>>>>> cases for them.
> >>>>>>>>
> >>>>>>>> Sergi
> >>>>>>>>
> >>>>>>>> 2017-04-09 17:00 GMT+03:00 Denis Magda <dm...@gridgain.com>:
> >>>>>>>>
> >>>>>>>>> Sergi, Vovan,
> >>>>>>>>>
> >>>>>>>>> Sorry for being annoying but I still didn't get an answer on
> >>> whether
> >>>>>> the
> >>>>>>>>> resolvers are the must for DML. The main reason why we made them
> up
> >>>>>> some
> >>>>>>>>> time ago is to support specific DML use cases. However I can't
> >>> recall
> >>>>>> the
> >>>>>>>>> use cases.
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Denis
> >>>>>>>>>
> >>>>>>>>> On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <
> >>>>>> sergi.vladykin@gmail.com
> >>>>>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Ok, we need to do 2 things here:
> >>>>>>>>>>
> >>>>>>>>>> 1. Drop the resolvers from the source code.
> >>>>>>>>>> 2. Write a good page in docs on "What makes a correct cache
> key".
> >>>>>>>>>>
> >>>>>>>>>> Who can do that?
> >>>>>>>>>>
> >>>>>>>>>> Sergi
> >>>>>>>>>>
> >>>>>>>>>> 2017-04-07 9:48 GMT+03:00 Sergi Vladykin <
> >>> sergi.vladykin@gmail.com
> >>>>> :
> >>>>>>>>>>
> >>>>>>>>>>> It is possible to try adding support of comparison to
> Resolvers,
> >>>> but
> >>>>>>>>> the
> >>>>>>>>>>> whole approach looks wrong and for now it is better to get rid
> of
> >>>> it
> >>>>>>>>>> while
> >>>>>>>>>>> we have a chance to break compatibility.
> >>>>>>>>>>>
> >>>>>>>>>>> Sergi
> >>>>>>>>>>>
> >>>>>>>>>>> 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
> >>>>>>>>>>> valentin.kulichenko@gmail.com>:
> >>>>>>>>>>>
> >>>>>>>>>>>> The discussion should've been started with that :) If
> supporting
> >>>>>>>>>> resolvers
> >>>>>>>>>>>> in new architecture is not possible or means too big effort,
> >>> then
> >>>>>>>> it's
> >>>>>>>>>>>> definitely not worth it.
> >>>>>>>>>>>>
> >>>>>>>>>>>> -Val
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <
> >>>>>>>> vozerov@gridgain.com
> >>>>>>>>>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Dima,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Yes, they may explode some internals of our indexes.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan" <
> >>>>>>>>>>>>> dsetrakyan@apache.org> написал:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Guys,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Isn't the main issue here that we cannot use the Identity
> >>>>>>>>> Resolvers
> >>>>>>>>>> in
> >>>>>>>>>>>>>> BTrees in the 2.0 version? If yes, then we have to remove
> them
> >>>>>>>> no
> >>>>>>>>>>>> matter
> >>>>>>>>>>>>>> what.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> D.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
> >>>>>>>>>>>> sergi.vladykin@gmail.com
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Binary key representation is stable when we always have
> equal
> >>>>>>>>>>>>> serialized
> >>>>>>>>>>>>>>> bytes when the original keys are equal.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Resolver allows you to have some extra info in the Key and
> >>>>>>>> equal
> >>>>>>>>>>>> Keys
> >>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>> be serialized into different bytes, which is wrong.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Look at the example what you can do with resolvers:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> We may have some data entry with fields a, b, c. Let's say
> >>> the
> >>>>>>>>>>>> unique
> >>>>>>>>>>>>>> part
> >>>>>>>>>>>>>>> here is `a` and it the only fields used in Key equals() and
> >>>>>>>>>>>> hashCode().
> >>>>>>>>>>>>>>> Still we may have the following layouts:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 1. Ka -> Vbc
> >>>>>>>>>>>>>>> 2. Kab -> Vc
> >>>>>>>>>>>>>>> 3. Kabc -> Boolean.TRUE
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> The only 1 is a correct layout, others are plain wrong
> >>>>>>>> variants
> >>>>>>>>>> (but
> >>>>>>>>>>>>> they
> >>>>>>>>>>>>>>> are still possible with Resolvers) because everything that
> >>>>>>>> does
> >>>>>>>>>> not
> >>>>>>>>>>>>> make
> >>>>>>>>>>>>>>> Key unique must be in Value.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> We want to clearly state that if you have something in Key,
> >>>>>>>> that
> >>>>>>>>>> is
> >>>>>>>>>>>> not
> >>>>>>>>>>>>>>> part of equals(), then the Key is invalid and that stuff
> must
> >>>>>>>> be
> >>>>>>>>>> in
> >>>>>>>>>>>>>> Value.
> >>>>>>>>>>>>>>> This allows us to rely on binary representation of a Key to
> >>> be
> >>>>>>>>>>>> stable
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>>> have some more optimizations and code simplifications with
> >>>>>>>>> respect
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>> assumptions.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Sergi
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
> >>>>>>>>>>>>>>> valentin.kulichenko@gmail.com>:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Even with my vast expirience I would never claim that I've
> >>>>>>>>> seen
> >>>>>>>>>>>>>>>> "everything" :)
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> What do you mean by stable binary key representation and
> how
> >>>>>>>>>>>>> resolvers
> >>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>> it unstable?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> -Val
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
> >>>>>>>>>>>>>> sergi.vladykin@gmail.com
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Val,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I know that you have really vast experience in Ignite
> >>>>>>>>>>>> deployments
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> probably saw everything that can happen. Did you ever see
> >>>>>>>>>>>> identity
> >>>>>>>>>>>>>>>>> resolvers use in real life? I guess no.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hibernate example is bad here, because if their key is
> >>>>>>>>>> unstable
> >>>>>>>>>>>>>> across
> >>>>>>>>>>>>>>>>> multiple JVMs, it means that it was not designed for
> >>>>>>>>>> distributed
> >>>>>>>>>>>>>>> caches a
> >>>>>>>>>>>>>>>>> priori.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Also knowing in advance about stable binary key
> >>>>>>>>> representation
> >>>>>>>>>>>>> allows
> >>>>>>>>>>>>>>> us
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> apply additional optimizations, like comparing keys
> >>>>>>>> without
> >>>>>>>>>>>>> detaching
> >>>>>>>>>>>>>>>> them
> >>>>>>>>>>>>>>>>> from offheap memory.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> We always will be able to add this stuff back if we see
> >>>>>>>>> users
> >>>>>>>>>>>>> really
> >>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>> it. Let's remove it for 2.0.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Sergi
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
> >>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com>:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Alex,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> To be honest, I don't understand the reasoning behind
> >>>>>>>> the
> >>>>>>>>>>>>> removal.
> >>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>>> resolvers provide good flexibility for different corner
> >>>>>>>>>> cases
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>> good thing to have them. Note that they can be applied
> >>>>>>>> not
> >>>>>>>>>>>> only
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> cache
> >>>>>>>>>>>>>>>>>> keys, but to any binary objects.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Hibernate issue is actually a good example of such use
> >>>>>>>>> case.
> >>>>>>>>>>>> The
> >>>>>>>>>>>>>> fact
> >>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>> we found an alternative solution doesn't actually mean
> >>>>>>>>>>>> anything,
> >>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>> what if this happened not in our module, but in user's
> >>>>>>>>>>>>> application?
> >>>>>>>>>>>>>>>>>> Unfortunately, we can't predict everything.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Error proneness is not a very strong argument either,
> >>>>>>>>>> because
> >>>>>>>>>>>> in
> >>>>>>>>>>>>> my
> >>>>>>>>>>>>>>>> view
> >>>>>>>>>>>>>>>>>> these resolvers are as much error prone as
> >>>>>>>> BinaryIdMapper,
> >>>>>>>>>> for
> >>>>>>>>>>>>>>> example.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> -Val
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
> >>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Denis,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Can you suggest a use-case where identity resolver is
> >>>>>>>>>> needed
> >>>>>>>>>>>>>> (given
> >>>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>> agree that a key must contain only valuable fields)?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 2017-04-05 22:08 GMT+03:00 Denis Magda <
> >>>>>>>>> dmagda@apache.org
> >>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Where do you want to remove the identity resolvers
> >>>>>>>>> from?
> >>>>>>>>>>>> If
> >>>>>>>>>>>>>> it’s
> >>>>>>>>>>>>>>>>>> related
> >>>>>>>>>>>>>>>>>>>> to the internals of Hibernate module then it’s fine
> >>>>>>>>> but
> >>>>>>>>>> if
> >>>>>>>>>>>>> you
> >>>>>>>>>>>>>>>>> suggest
> >>>>>>>>>>>>>>>>>>>> removing identity resolvers public interfaces then
> >>>>>>>> it
> >>>>>>>>>>>> might
> >>>>>>>>>>>>> be
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>> haste
> >>>>>>>>>>>>>>>>>>>> decision.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> —
> >>>>>>>>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
> >>>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> +1, I see no other reasons to keep it.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
> >>>>>>>>>>>>>>>>> sergi.vladykin@gmail.com
> >>>>>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> +1
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Lets drop them.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Sergi
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
> >>>>>>>>>>>>>>>>>>>>>> dmitriy.govorukhin@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Hi guys, i implemented proxy for IgniteCache in
> >>>>>>>>>>>> hibernate
> >>>>>>>>>>>>>>>>>>> integration,
> >>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>> proxy transformate cacheKey to our key wrapper,
> >>>>>>>>>> leaves
> >>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>> required
> >>>>>>>>>>>>>>>>>>>>>>> field. I think we can remove identity resolve,
> >>>>>>>> it
> >>>>>>>>>>>> should
> >>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>> broke
> >>>>>>>>>>>>>>>>>>>>>>> integration with hibernate. Any objections?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin
> >>>>>>>>>> Kulichenko
> >>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I'm not saying there is no alternative
> >>>>>>>> solution.
> >>>>>>>>>> But
> >>>>>>>>>>>>> let's
> >>>>>>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> prove that it works first, and remove resolvers
> >>>>>>>>>> only
> >>>>>>>>>>>>> after
> >>>>>>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> -Val
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi
> >>>>>>>> Vladykin
> >>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>> sergi.vladykin@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Guys, nothing is impossible if you know a bit
> >>>>>>>>>> about
> >>>>>>>>>>>>>>>> reflection
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>> Java
> >>>>>>>>>>>>>>>>>>>>>>> :)
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> We had a look at the CacheKey class and it is
> >>>>>>>>>> easily
> >>>>>>>>>>>>>>>>> replaceable.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Sergi
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
> >>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org
> >>>>>>>>>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin
> >>>>>>>>>>>> Kulichenko
> >>>>>>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> "Hibernate key" is the CacheKey class I was
> >>>>>>>>>>>> referring
> >>>>>>>>>>>>>> to.
> >>>>>>>>>>>>>>>>> It's
> >>>>>>>>>>>>>>>>>>>>>>>> provided
> >>>>>>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hibernate, not by user and not by us. So I'm
> >>>>>>>>> not
> >>>>>>>>>>>> sure
> >>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>> possible
> >>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>> replace it.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> If it is impossible to replace or get rid of
> >>>>>>>>> the
> >>>>>>>>>>>>>> Hibernate
> >>>>>>>>>>>>>>>>> key,
> >>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>>>>> discussion valid at all?
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>
> >
>
>
--
Best regards,
Andrey V. Mashenkov
Re: Stable binary key representation
Posted by Denis Magda <dm...@apache.org>.
At all, guys, BinaryIdentityResolvers were discontinued but the ticket [1] that had triggered the discussion has not been fixed yet.
It must be fixed in 2.0 otherwise Hibernate integration can be considered broken in 2.0 because the initial workaround was based on the resolvers.
Andrey M., will you finalize it. Alex G. and Sergi can suggest non-resolvers based solution.
[1] https://issues.apache.org/jira/browse/IGNITE-3429 <https://issues.apache.org/jira/browse/IGNITE-3429>
—
Denis
> On Apr 11, 2017, at 12:06 PM, Denis Magda <dm...@apache.org> wrote:
>
> I don’t see either unless a key’s field is of a float type. However, it sounds like an artificial use case.
>
> Thanks for the details.
>
> —
> Denis
>
>> On Apr 11, 2017, at 11:50 AM, Dmitriy Setrakyan <ds...@apache.org> wrote:
>>
>> Denis, I think it is important that we know which specific field to use for
>> the affinity resolution, but I don't see any issue in using both, primary
>> and foreign keys, for hashcode and equality. Do you?
>>
>> D.
>>
>> On Tue, Apr 11, 2017 at 11:46 AM, Igor Sapego <is...@apache.org> wrote:
>>
>>> Denis,
>>>
>>> The whole binary representation of the object is used now
>>> for hash code generation and equality comparison. So the
>>> answer - all fields are used for this.
>>>
>>> Best Regards,
>>> Igor
>>>
>>> On Mon, Apr 10, 2017 at 9:50 PM, Denis Magda <dm...@apache.org> wrote:
>>>
>>>> Considering this simple example
>>>>
>>>> INSERT (id, orgId, name, age, address) into Person…
>>>>
>>>> where id and orgId define Person’s affinity key - PersonKey(id, orgId)
>>>>
>>>> How do we know which fields to use for hash code generation and equality
>>>> comparison? QueryEntity?
>>>>
>>>> No, it’s unclear how to document it properly.
>>>>
>>>> —
>>>> Denis
>>>>
>>>>> On Apr 10, 2017, at 11:14 AM, Vladimir Ozerov <vo...@gridgain.com>
>>>> wrote:
>>>>>
>>>>> There is no more such resolver. It was removed.
>>>>>
>>>>> On Mon, Apr 10, 2017 at 8:58 PM, Denis Magda <dm...@apache.org>
>>> wrote:
>>>>>
>>>>>> Vovan,
>>>>>>
>>>>>> Before I fix the documentation, what’t the replacement for
>>>>>> BinaryFieldIdentiyResolver we used to define field for hash code
>>>>>> calculation and equality comparison when DML statements are used?
>>>>>> https://apacheignite.readme.io/docs/binary-marshaller#
>>>>>> section-binary-field-identity-resolver <https://apacheignite.readme.
>>>>>> io/docs/binary-marshaller#section-binary-field-identity-resolver>
>>>>>>
>>>>>> —
>>>>>> Denis
>>>>>>
>>>>>>> On Apr 9, 2017, at 7:39 AM, Vladimir Ozerov <vo...@gridgain.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> Resolvers were essential for DML because we had broken comparison
>>>>>> semantics
>>>>>>> of binary objects. This is not the case now.
>>>>>>>
>>>>>>> Resolver as a whole is normal practice. E.g. it is implemented in
>>> .NET
>>>> on
>>>>>>> core language level and widely used in many cases. Hazelcast has it
>>> as
>>>>>> well
>>>>>>> AFAIK. So it is wrong to think that the whole idea is useless. Think
>>> of
>>>>>> it
>>>>>>> as a comparator's brother.
>>>>>>>
>>>>>>> The only reason why we need to remove it is missing hash index in new
>>>>>>> architecture. It makes sense, as it is better to have AI 2.0 without
>>>>>> them,
>>>>>>> than no AI 2.0 :-)
>>>>>>>
>>>>>>> 09 апр. 2017 г. 17:31 пользователь "Sergi Vladykin" <
>>>>>>> sergi.vladykin@gmail.com> написал:
>>>>>>>
>>>>>>>> I guess Resolvers were added to DML just because they already
>>> existed
>>>>>> since
>>>>>>>> 1.9 and we were forced to support them in all the parts of our
>>>> product.
>>>>>>>>
>>>>>>>> We have to stop this practice to add features without clear real
>>> life
>>>>>> use
>>>>>>>> cases for them.
>>>>>>>>
>>>>>>>> Sergi
>>>>>>>>
>>>>>>>> 2017-04-09 17:00 GMT+03:00 Denis Magda <dm...@gridgain.com>:
>>>>>>>>
>>>>>>>>> Sergi, Vovan,
>>>>>>>>>
>>>>>>>>> Sorry for being annoying but I still didn't get an answer on
>>> whether
>>>>>> the
>>>>>>>>> resolvers are the must for DML. The main reason why we made them up
>>>>>> some
>>>>>>>>> time ago is to support specific DML use cases. However I can't
>>> recall
>>>>>> the
>>>>>>>>> use cases.
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Denis
>>>>>>>>>
>>>>>>>>> On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <
>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Ok, we need to do 2 things here:
>>>>>>>>>>
>>>>>>>>>> 1. Drop the resolvers from the source code.
>>>>>>>>>> 2. Write a good page in docs on "What makes a correct cache key".
>>>>>>>>>>
>>>>>>>>>> Who can do that?
>>>>>>>>>>
>>>>>>>>>> Sergi
>>>>>>>>>>
>>>>>>>>>> 2017-04-07 9:48 GMT+03:00 Sergi Vladykin <
>>> sergi.vladykin@gmail.com
>>>>> :
>>>>>>>>>>
>>>>>>>>>>> It is possible to try adding support of comparison to Resolvers,
>>>> but
>>>>>>>>> the
>>>>>>>>>>> whole approach looks wrong and for now it is better to get rid of
>>>> it
>>>>>>>>>> while
>>>>>>>>>>> we have a chance to break compatibility.
>>>>>>>>>>>
>>>>>>>>>>> Sergi
>>>>>>>>>>>
>>>>>>>>>>> 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
>>>>>>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>>>>>>
>>>>>>>>>>>> The discussion should've been started with that :) If supporting
>>>>>>>>>> resolvers
>>>>>>>>>>>> in new architecture is not possible or means too big effort,
>>> then
>>>>>>>> it's
>>>>>>>>>>>> definitely not worth it.
>>>>>>>>>>>>
>>>>>>>>>>>> -Val
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <
>>>>>>>> vozerov@gridgain.com
>>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Dima,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yes, they may explode some internals of our indexes.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan" <
>>>>>>>>>>>>> dsetrakyan@apache.org> написал:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Guys,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Isn't the main issue here that we cannot use the Identity
>>>>>>>>> Resolvers
>>>>>>>>>> in
>>>>>>>>>>>>>> BTrees in the 2.0 version? If yes, then we have to remove them
>>>>>>>> no
>>>>>>>>>>>> matter
>>>>>>>>>>>>>> what.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
>>>>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Binary key representation is stable when we always have equal
>>>>>>>>>>>>> serialized
>>>>>>>>>>>>>>> bytes when the original keys are equal.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Resolver allows you to have some extra info in the Key and
>>>>>>>> equal
>>>>>>>>>>>> Keys
>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>> be serialized into different bytes, which is wrong.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Look at the example what you can do with resolvers:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We may have some data entry with fields a, b, c. Let's say
>>> the
>>>>>>>>>>>> unique
>>>>>>>>>>>>>> part
>>>>>>>>>>>>>>> here is `a` and it the only fields used in Key equals() and
>>>>>>>>>>>> hashCode().
>>>>>>>>>>>>>>> Still we may have the following layouts:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1. Ka -> Vbc
>>>>>>>>>>>>>>> 2. Kab -> Vc
>>>>>>>>>>>>>>> 3. Kabc -> Boolean.TRUE
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The only 1 is a correct layout, others are plain wrong
>>>>>>>> variants
>>>>>>>>>> (but
>>>>>>>>>>>>> they
>>>>>>>>>>>>>>> are still possible with Resolvers) because everything that
>>>>>>>> does
>>>>>>>>>> not
>>>>>>>>>>>>> make
>>>>>>>>>>>>>>> Key unique must be in Value.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We want to clearly state that if you have something in Key,
>>>>>>>> that
>>>>>>>>>> is
>>>>>>>>>>>> not
>>>>>>>>>>>>>>> part of equals(), then the Key is invalid and that stuff must
>>>>>>>> be
>>>>>>>>>> in
>>>>>>>>>>>>>> Value.
>>>>>>>>>>>>>>> This allows us to rely on binary representation of a Key to
>>> be
>>>>>>>>>>>> stable
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> have some more optimizations and code simplifications with
>>>>>>>>> respect
>>>>>>>>>>>> to
>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>> assumptions.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Even with my vast expirience I would never claim that I've
>>>>>>>>> seen
>>>>>>>>>>>>>>>> "everything" :)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> What do you mean by stable binary key representation and how
>>>>>>>>>>>>> resolvers
>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>> it unstable?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -Val
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
>>>>>>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Val,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I know that you have really vast experience in Ignite
>>>>>>>>>>>> deployments
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> probably saw everything that can happen. Did you ever see
>>>>>>>>>>>> identity
>>>>>>>>>>>>>>>>> resolvers use in real life? I guess no.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hibernate example is bad here, because if their key is
>>>>>>>>>> unstable
>>>>>>>>>>>>>> across
>>>>>>>>>>>>>>>>> multiple JVMs, it means that it was not designed for
>>>>>>>>>> distributed
>>>>>>>>>>>>>>> caches a
>>>>>>>>>>>>>>>>> priori.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Also knowing in advance about stable binary key
>>>>>>>>> representation
>>>>>>>>>>>>> allows
>>>>>>>>>>>>>>> us
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> apply additional optimizations, like comparing keys
>>>>>>>> without
>>>>>>>>>>>>> detaching
>>>>>>>>>>>>>>>> them
>>>>>>>>>>>>>>>>> from offheap memory.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> We always will be able to add this stuff back if we see
>>>>>>>>> users
>>>>>>>>>>>>> really
>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>> it. Let's remove it for 2.0.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Alex,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> To be honest, I don't understand the reasoning behind
>>>>>>>> the
>>>>>>>>>>>>> removal.
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>> resolvers provide good flexibility for different corner
>>>>>>>>>> cases
>>>>>>>>>>>> and
>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> good thing to have them. Note that they can be applied
>>>>>>>> not
>>>>>>>>>>>> only
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> cache
>>>>>>>>>>>>>>>>>> keys, but to any binary objects.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hibernate issue is actually a good example of such use
>>>>>>>>> case.
>>>>>>>>>>>> The
>>>>>>>>>>>>>> fact
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>> we found an alternative solution doesn't actually mean
>>>>>>>>>>>> anything,
>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>> what if this happened not in our module, but in user's
>>>>>>>>>>>>> application?
>>>>>>>>>>>>>>>>>> Unfortunately, we can't predict everything.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Error proneness is not a very strong argument either,
>>>>>>>>>> because
>>>>>>>>>>>> in
>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>> view
>>>>>>>>>>>>>>>>>> these resolvers are as much error prone as
>>>>>>>> BinaryIdMapper,
>>>>>>>>>> for
>>>>>>>>>>>>>>> example.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> -Val
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
>>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Denis,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Can you suggest a use-case where identity resolver is
>>>>>>>>>> needed
>>>>>>>>>>>>>> (given
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>> agree that a key must contain only valuable fields)?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 2017-04-05 22:08 GMT+03:00 Denis Magda <
>>>>>>>>> dmagda@apache.org
>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Where do you want to remove the identity resolvers
>>>>>>>>> from?
>>>>>>>>>>>> If
>>>>>>>>>>>>>> it’s
>>>>>>>>>>>>>>>>>> related
>>>>>>>>>>>>>>>>>>>> to the internals of Hibernate module then it’s fine
>>>>>>>>> but
>>>>>>>>>> if
>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>> suggest
>>>>>>>>>>>>>>>>>>>> removing identity resolvers public interfaces then
>>>>>>>> it
>>>>>>>>>>>> might
>>>>>>>>>>>>> be
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> haste
>>>>>>>>>>>>>>>>>>>> decision.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
>>>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> +1, I see no other reasons to keep it.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
>>>>>>>>>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Lets drop them.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
>>>>>>>>>>>>>>>>>>>>>> dmitriy.govorukhin@gmail.com>
>>>>>>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi guys, i implemented proxy for IgniteCache in
>>>>>>>>>>>> hibernate
>>>>>>>>>>>>>>>>>>> integration,
>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>> proxy transformate cacheKey to our key wrapper,
>>>>>>>>>> leaves
>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>> required
>>>>>>>>>>>>>>>>>>>>>>> field. I think we can remove identity resolve,
>>>>>>>> it
>>>>>>>>>>>> should
>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>> broke
>>>>>>>>>>>>>>>>>>>>>>> integration with hibernate. Any objections?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin
>>>>>>>>>> Kulichenko
>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I'm not saying there is no alternative
>>>>>>>> solution.
>>>>>>>>>> But
>>>>>>>>>>>>> let's
>>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> prove that it works first, and remove resolvers
>>>>>>>>>> only
>>>>>>>>>>>>> after
>>>>>>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> -Val
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi
>>>>>>>> Vladykin
>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Guys, nothing is impossible if you know a bit
>>>>>>>>>> about
>>>>>>>>>>>>>>>> reflection
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> Java
>>>>>>>>>>>>>>>>>>>>>>> :)
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> We had a look at the CacheKey class and it is
>>>>>>>>>> easily
>>>>>>>>>>>>>>>>> replaceable.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
>>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org
>>>>>>>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin
>>>>>>>>>>>> Kulichenko
>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> "Hibernate key" is the CacheKey class I was
>>>>>>>>>>>> referring
>>>>>>>>>>>>>> to.
>>>>>>>>>>>>>>>>> It's
>>>>>>>>>>>>>>>>>>>>>>>> provided
>>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hibernate, not by user and not by us. So I'm
>>>>>>>>> not
>>>>>>>>>>>> sure
>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>> possible
>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>> replace it.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> If it is impossible to replace or get rid of
>>>>>>>>> the
>>>>>>>>>>>>>> Hibernate
>>>>>>>>>>>>>>>>> key,
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>>> discussion valid at all?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>>
>
Re: Stable binary key representation
Posted by Denis Magda <dm...@apache.org>.
I don’t see either unless a key’s field is of a float type. However, it sounds like an artificial use case.
Thanks for the details.
—
Denis
> On Apr 11, 2017, at 11:50 AM, Dmitriy Setrakyan <ds...@apache.org> wrote:
>
> Denis, I think it is important that we know which specific field to use for
> the affinity resolution, but I don't see any issue in using both, primary
> and foreign keys, for hashcode and equality. Do you?
>
> D.
>
> On Tue, Apr 11, 2017 at 11:46 AM, Igor Sapego <is...@apache.org> wrote:
>
>> Denis,
>>
>> The whole binary representation of the object is used now
>> for hash code generation and equality comparison. So the
>> answer - all fields are used for this.
>>
>> Best Regards,
>> Igor
>>
>> On Mon, Apr 10, 2017 at 9:50 PM, Denis Magda <dm...@apache.org> wrote:
>>
>>> Considering this simple example
>>>
>>> INSERT (id, orgId, name, age, address) into Person…
>>>
>>> where id and orgId define Person’s affinity key - PersonKey(id, orgId)
>>>
>>> How do we know which fields to use for hash code generation and equality
>>> comparison? QueryEntity?
>>>
>>> No, it’s unclear how to document it properly.
>>>
>>> —
>>> Denis
>>>
>>>> On Apr 10, 2017, at 11:14 AM, Vladimir Ozerov <vo...@gridgain.com>
>>> wrote:
>>>>
>>>> There is no more such resolver. It was removed.
>>>>
>>>> On Mon, Apr 10, 2017 at 8:58 PM, Denis Magda <dm...@apache.org>
>> wrote:
>>>>
>>>>> Vovan,
>>>>>
>>>>> Before I fix the documentation, what’t the replacement for
>>>>> BinaryFieldIdentiyResolver we used to define field for hash code
>>>>> calculation and equality comparison when DML statements are used?
>>>>> https://apacheignite.readme.io/docs/binary-marshaller#
>>>>> section-binary-field-identity-resolver <https://apacheignite.readme.
>>>>> io/docs/binary-marshaller#section-binary-field-identity-resolver>
>>>>>
>>>>> —
>>>>> Denis
>>>>>
>>>>>> On Apr 9, 2017, at 7:39 AM, Vladimir Ozerov <vo...@gridgain.com>
>>>>> wrote:
>>>>>>
>>>>>> Resolvers were essential for DML because we had broken comparison
>>>>> semantics
>>>>>> of binary objects. This is not the case now.
>>>>>>
>>>>>> Resolver as a whole is normal practice. E.g. it is implemented in
>> .NET
>>> on
>>>>>> core language level and widely used in many cases. Hazelcast has it
>> as
>>>>> well
>>>>>> AFAIK. So it is wrong to think that the whole idea is useless. Think
>> of
>>>>> it
>>>>>> as a comparator's brother.
>>>>>>
>>>>>> The only reason why we need to remove it is missing hash index in new
>>>>>> architecture. It makes sense, as it is better to have AI 2.0 without
>>>>> them,
>>>>>> than no AI 2.0 :-)
>>>>>>
>>>>>> 09 апр. 2017 г. 17:31 пользователь "Sergi Vladykin" <
>>>>>> sergi.vladykin@gmail.com> написал:
>>>>>>
>>>>>>> I guess Resolvers were added to DML just because they already
>> existed
>>>>> since
>>>>>>> 1.9 and we were forced to support them in all the parts of our
>>> product.
>>>>>>>
>>>>>>> We have to stop this practice to add features without clear real
>> life
>>>>> use
>>>>>>> cases for them.
>>>>>>>
>>>>>>> Sergi
>>>>>>>
>>>>>>> 2017-04-09 17:00 GMT+03:00 Denis Magda <dm...@gridgain.com>:
>>>>>>>
>>>>>>>> Sergi, Vovan,
>>>>>>>>
>>>>>>>> Sorry for being annoying but I still didn't get an answer on
>> whether
>>>>> the
>>>>>>>> resolvers are the must for DML. The main reason why we made them up
>>>>> some
>>>>>>>> time ago is to support specific DML use cases. However I can't
>> recall
>>>>> the
>>>>>>>> use cases.
>>>>>>>>
>>>>>>>> --
>>>>>>>> Denis
>>>>>>>>
>>>>>>>> On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <
>>>>> sergi.vladykin@gmail.com
>>>>>>>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Ok, we need to do 2 things here:
>>>>>>>>>
>>>>>>>>> 1. Drop the resolvers from the source code.
>>>>>>>>> 2. Write a good page in docs on "What makes a correct cache key".
>>>>>>>>>
>>>>>>>>> Who can do that?
>>>>>>>>>
>>>>>>>>> Sergi
>>>>>>>>>
>>>>>>>>> 2017-04-07 9:48 GMT+03:00 Sergi Vladykin <
>> sergi.vladykin@gmail.com
>>>> :
>>>>>>>>>
>>>>>>>>>> It is possible to try adding support of comparison to Resolvers,
>>> but
>>>>>>>> the
>>>>>>>>>> whole approach looks wrong and for now it is better to get rid of
>>> it
>>>>>>>>> while
>>>>>>>>>> we have a chance to break compatibility.
>>>>>>>>>>
>>>>>>>>>> Sergi
>>>>>>>>>>
>>>>>>>>>> 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
>>>>>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>>>>>
>>>>>>>>>>> The discussion should've been started with that :) If supporting
>>>>>>>>> resolvers
>>>>>>>>>>> in new architecture is not possible or means too big effort,
>> then
>>>>>>> it's
>>>>>>>>>>> definitely not worth it.
>>>>>>>>>>>
>>>>>>>>>>> -Val
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <
>>>>>>> vozerov@gridgain.com
>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Dima,
>>>>>>>>>>>>
>>>>>>>>>>>> Yes, they may explode some internals of our indexes.
>>>>>>>>>>>>
>>>>>>>>>>>> 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan" <
>>>>>>>>>>>> dsetrakyan@apache.org> написал:
>>>>>>>>>>>>
>>>>>>>>>>>>> Guys,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Isn't the main issue here that we cannot use the Identity
>>>>>>>> Resolvers
>>>>>>>>> in
>>>>>>>>>>>>> BTrees in the 2.0 version? If yes, then we have to remove them
>>>>>>> no
>>>>>>>>>>> matter
>>>>>>>>>>>>> what.
>>>>>>>>>>>>>
>>>>>>>>>>>>> D.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
>>>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Binary key representation is stable when we always have equal
>>>>>>>>>>>> serialized
>>>>>>>>>>>>>> bytes when the original keys are equal.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Resolver allows you to have some extra info in the Key and
>>>>>>> equal
>>>>>>>>>>> Keys
>>>>>>>>>>>>> will
>>>>>>>>>>>>>> be serialized into different bytes, which is wrong.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Look at the example what you can do with resolvers:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We may have some data entry with fields a, b, c. Let's say
>> the
>>>>>>>>>>> unique
>>>>>>>>>>>>> part
>>>>>>>>>>>>>> here is `a` and it the only fields used in Key equals() and
>>>>>>>>>>> hashCode().
>>>>>>>>>>>>>> Still we may have the following layouts:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1. Ka -> Vbc
>>>>>>>>>>>>>> 2. Kab -> Vc
>>>>>>>>>>>>>> 3. Kabc -> Boolean.TRUE
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The only 1 is a correct layout, others are plain wrong
>>>>>>> variants
>>>>>>>>> (but
>>>>>>>>>>>> they
>>>>>>>>>>>>>> are still possible with Resolvers) because everything that
>>>>>>> does
>>>>>>>>> not
>>>>>>>>>>>> make
>>>>>>>>>>>>>> Key unique must be in Value.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We want to clearly state that if you have something in Key,
>>>>>>> that
>>>>>>>>> is
>>>>>>>>>>> not
>>>>>>>>>>>>>> part of equals(), then the Key is invalid and that stuff must
>>>>>>> be
>>>>>>>>> in
>>>>>>>>>>>>> Value.
>>>>>>>>>>>>>> This allows us to rely on binary representation of a Key to
>> be
>>>>>>>>>>> stable
>>>>>>>>>>>> and
>>>>>>>>>>>>>> have some more optimizations and code simplifications with
>>>>>>>> respect
>>>>>>>>>>> to
>>>>>>>>>>>>> these
>>>>>>>>>>>>>> assumptions.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
>>>>>>>>>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Even with my vast expirience I would never claim that I've
>>>>>>>> seen
>>>>>>>>>>>>>>> "everything" :)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> What do you mean by stable binary key representation and how
>>>>>>>>>>>> resolvers
>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>> it unstable?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -Val
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
>>>>>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Val,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I know that you have really vast experience in Ignite
>>>>>>>>>>> deployments
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> probably saw everything that can happen. Did you ever see
>>>>>>>>>>> identity
>>>>>>>>>>>>>>>> resolvers use in real life? I guess no.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hibernate example is bad here, because if their key is
>>>>>>>>> unstable
>>>>>>>>>>>>> across
>>>>>>>>>>>>>>>> multiple JVMs, it means that it was not designed for
>>>>>>>>> distributed
>>>>>>>>>>>>>> caches a
>>>>>>>>>>>>>>>> priori.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Also knowing in advance about stable binary key
>>>>>>>> representation
>>>>>>>>>>>> allows
>>>>>>>>>>>>>> us
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> apply additional optimizations, like comparing keys
>>>>>>> without
>>>>>>>>>>>> detaching
>>>>>>>>>>>>>>> them
>>>>>>>>>>>>>>>> from offheap memory.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We always will be able to add this stuff back if we see
>>>>>>>> users
>>>>>>>>>>>> really
>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>> it. Let's remove it for 2.0.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Alex,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To be honest, I don't understand the reasoning behind
>>>>>>> the
>>>>>>>>>>>> removal.
>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>> resolvers provide good flexibility for different corner
>>>>>>>>> cases
>>>>>>>>>>> and
>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> good thing to have them. Note that they can be applied
>>>>>>> not
>>>>>>>>>>> only
>>>>>>>>>>>> to
>>>>>>>>>>>>>>> cache
>>>>>>>>>>>>>>>>> keys, but to any binary objects.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hibernate issue is actually a good example of such use
>>>>>>>> case.
>>>>>>>>>>> The
>>>>>>>>>>>>> fact
>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> we found an alternative solution doesn't actually mean
>>>>>>>>>>> anything,
>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>> what if this happened not in our module, but in user's
>>>>>>>>>>>> application?
>>>>>>>>>>>>>>>>> Unfortunately, we can't predict everything.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Error proneness is not a very strong argument either,
>>>>>>>>> because
>>>>>>>>>>> in
>>>>>>>>>>>> my
>>>>>>>>>>>>>>> view
>>>>>>>>>>>>>>>>> these resolvers are as much error prone as
>>>>>>> BinaryIdMapper,
>>>>>>>>> for
>>>>>>>>>>>>>> example.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> -Val
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Denis,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Can you suggest a use-case where identity resolver is
>>>>>>>>> needed
>>>>>>>>>>>>> (given
>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>> agree that a key must contain only valuable fields)?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 2017-04-05 22:08 GMT+03:00 Denis Magda <
>>>>>>>> dmagda@apache.org
>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Where do you want to remove the identity resolvers
>>>>>>>> from?
>>>>>>>>>>> If
>>>>>>>>>>>>> it’s
>>>>>>>>>>>>>>>>> related
>>>>>>>>>>>>>>>>>>> to the internals of Hibernate module then it’s fine
>>>>>>>> but
>>>>>>>>> if
>>>>>>>>>>>> you
>>>>>>>>>>>>>>>> suggest
>>>>>>>>>>>>>>>>>>> removing identity resolvers public interfaces then
>>>>>>> it
>>>>>>>>>>> might
>>>>>>>>>>>> be
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> haste
>>>>>>>>>>>>>>>>>>> decision.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
>>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> +1, I see no other reasons to keep it.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
>>>>>>>>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Lets drop them.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
>>>>>>>>>>>>>>>>>>>>> dmitriy.govorukhin@gmail.com>
>>>>>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hi guys, i implemented proxy for IgniteCache in
>>>>>>>>>>> hibernate
>>>>>>>>>>>>>>>>>> integration,
>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>> proxy transformate cacheKey to our key wrapper,
>>>>>>>>> leaves
>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>> required
>>>>>>>>>>>>>>>>>>>>>> field. I think we can remove identity resolve,
>>>>>>> it
>>>>>>>>>>> should
>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>> broke
>>>>>>>>>>>>>>>>>>>>>> integration with hibernate. Any objections?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin
>>>>>>>>> Kulichenko
>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I'm not saying there is no alternative
>>>>>>> solution.
>>>>>>>>> But
>>>>>>>>>>>> let's
>>>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> prove that it works first, and remove resolvers
>>>>>>>>> only
>>>>>>>>>>>> after
>>>>>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> -Val
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi
>>>>>>> Vladykin
>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Guys, nothing is impossible if you know a bit
>>>>>>>>> about
>>>>>>>>>>>>>>> reflection
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>> Java
>>>>>>>>>>>>>>>>>>>>>> :)
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> We had a look at the CacheKey class and it is
>>>>>>>>> easily
>>>>>>>>>>>>>>>> replaceable.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org
>>>>>>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin
>>>>>>>>>>> Kulichenko
>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> "Hibernate key" is the CacheKey class I was
>>>>>>>>>>> referring
>>>>>>>>>>>>> to.
>>>>>>>>>>>>>>>> It's
>>>>>>>>>>>>>>>>>>>>>>> provided
>>>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>>>>> Hibernate, not by user and not by us. So I'm
>>>>>>>> not
>>>>>>>>>>> sure
>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>> possible
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> replace it.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> If it is impossible to replace or get rid of
>>>>>>>> the
>>>>>>>>>>>>> Hibernate
>>>>>>>>>>>>>>>> key,
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>>>> discussion valid at all?
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>
>>>
>>
Re: Stable binary key representation
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Denis, I think it is important that we know which specific field to use for
the affinity resolution, but I don't see any issue in using both, primary
and foreign keys, for hashcode and equality. Do you?
D.
On Tue, Apr 11, 2017 at 11:46 AM, Igor Sapego <is...@apache.org> wrote:
> Denis,
>
> The whole binary representation of the object is used now
> for hash code generation and equality comparison. So the
> answer - all fields are used for this.
>
> Best Regards,
> Igor
>
> On Mon, Apr 10, 2017 at 9:50 PM, Denis Magda <dm...@apache.org> wrote:
>
> > Considering this simple example
> >
> > INSERT (id, orgId, name, age, address) into Person…
> >
> > where id and orgId define Person’s affinity key - PersonKey(id, orgId)
> >
> > How do we know which fields to use for hash code generation and equality
> > comparison? QueryEntity?
> >
> > No, it’s unclear how to document it properly.
> >
> > —
> > Denis
> >
> > > On Apr 10, 2017, at 11:14 AM, Vladimir Ozerov <vo...@gridgain.com>
> > wrote:
> > >
> > > There is no more such resolver. It was removed.
> > >
> > > On Mon, Apr 10, 2017 at 8:58 PM, Denis Magda <dm...@apache.org>
> wrote:
> > >
> > >> Vovan,
> > >>
> > >> Before I fix the documentation, what’t the replacement for
> > >> BinaryFieldIdentiyResolver we used to define field for hash code
> > >> calculation and equality comparison when DML statements are used?
> > >> https://apacheignite.readme.io/docs/binary-marshaller#
> > >> section-binary-field-identity-resolver <https://apacheignite.readme.
> > >> io/docs/binary-marshaller#section-binary-field-identity-resolver>
> > >>
> > >> —
> > >> Denis
> > >>
> > >>> On Apr 9, 2017, at 7:39 AM, Vladimir Ozerov <vo...@gridgain.com>
> > >> wrote:
> > >>>
> > >>> Resolvers were essential for DML because we had broken comparison
> > >> semantics
> > >>> of binary objects. This is not the case now.
> > >>>
> > >>> Resolver as a whole is normal practice. E.g. it is implemented in
> .NET
> > on
> > >>> core language level and widely used in many cases. Hazelcast has it
> as
> > >> well
> > >>> AFAIK. So it is wrong to think that the whole idea is useless. Think
> of
> > >> it
> > >>> as a comparator's brother.
> > >>>
> > >>> The only reason why we need to remove it is missing hash index in new
> > >>> architecture. It makes sense, as it is better to have AI 2.0 without
> > >> them,
> > >>> than no AI 2.0 :-)
> > >>>
> > >>> 09 апр. 2017 г. 17:31 пользователь "Sergi Vladykin" <
> > >>> sergi.vladykin@gmail.com> написал:
> > >>>
> > >>>> I guess Resolvers were added to DML just because they already
> existed
> > >> since
> > >>>> 1.9 and we were forced to support them in all the parts of our
> > product.
> > >>>>
> > >>>> We have to stop this practice to add features without clear real
> life
> > >> use
> > >>>> cases for them.
> > >>>>
> > >>>> Sergi
> > >>>>
> > >>>> 2017-04-09 17:00 GMT+03:00 Denis Magda <dm...@gridgain.com>:
> > >>>>
> > >>>>> Sergi, Vovan,
> > >>>>>
> > >>>>> Sorry for being annoying but I still didn't get an answer on
> whether
> > >> the
> > >>>>> resolvers are the must for DML. The main reason why we made them up
> > >> some
> > >>>>> time ago is to support specific DML use cases. However I can't
> recall
> > >> the
> > >>>>> use cases.
> > >>>>>
> > >>>>> --
> > >>>>> Denis
> > >>>>>
> > >>>>> On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <
> > >> sergi.vladykin@gmail.com
> > >>>>>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Ok, we need to do 2 things here:
> > >>>>>>
> > >>>>>> 1. Drop the resolvers from the source code.
> > >>>>>> 2. Write a good page in docs on "What makes a correct cache key".
> > >>>>>>
> > >>>>>> Who can do that?
> > >>>>>>
> > >>>>>> Sergi
> > >>>>>>
> > >>>>>> 2017-04-07 9:48 GMT+03:00 Sergi Vladykin <
> sergi.vladykin@gmail.com
> > >:
> > >>>>>>
> > >>>>>>> It is possible to try adding support of comparison to Resolvers,
> > but
> > >>>>> the
> > >>>>>>> whole approach looks wrong and for now it is better to get rid of
> > it
> > >>>>>> while
> > >>>>>>> we have a chance to break compatibility.
> > >>>>>>>
> > >>>>>>> Sergi
> > >>>>>>>
> > >>>>>>> 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
> > >>>>>>> valentin.kulichenko@gmail.com>:
> > >>>>>>>
> > >>>>>>>> The discussion should've been started with that :) If supporting
> > >>>>>> resolvers
> > >>>>>>>> in new architecture is not possible or means too big effort,
> then
> > >>>> it's
> > >>>>>>>> definitely not worth it.
> > >>>>>>>>
> > >>>>>>>> -Val
> > >>>>>>>>
> > >>>>>>>> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <
> > >>>> vozerov@gridgain.com
> > >>>>>>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Dima,
> > >>>>>>>>>
> > >>>>>>>>> Yes, they may explode some internals of our indexes.
> > >>>>>>>>>
> > >>>>>>>>> 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan" <
> > >>>>>>>>> dsetrakyan@apache.org> написал:
> > >>>>>>>>>
> > >>>>>>>>>> Guys,
> > >>>>>>>>>>
> > >>>>>>>>>> Isn't the main issue here that we cannot use the Identity
> > >>>>> Resolvers
> > >>>>>> in
> > >>>>>>>>>> BTrees in the 2.0 version? If yes, then we have to remove them
> > >>>> no
> > >>>>>>>> matter
> > >>>>>>>>>> what.
> > >>>>>>>>>>
> > >>>>>>>>>> D.
> > >>>>>>>>>>
> > >>>>>>>>>> On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
> > >>>>>>>> sergi.vladykin@gmail.com
> > >>>>>>>>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Binary key representation is stable when we always have equal
> > >>>>>>>>> serialized
> > >>>>>>>>>>> bytes when the original keys are equal.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Resolver allows you to have some extra info in the Key and
> > >>>> equal
> > >>>>>>>> Keys
> > >>>>>>>>>> will
> > >>>>>>>>>>> be serialized into different bytes, which is wrong.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Look at the example what you can do with resolvers:
> > >>>>>>>>>>>
> > >>>>>>>>>>> We may have some data entry with fields a, b, c. Let's say
> the
> > >>>>>>>> unique
> > >>>>>>>>>> part
> > >>>>>>>>>>> here is `a` and it the only fields used in Key equals() and
> > >>>>>>>> hashCode().
> > >>>>>>>>>>> Still we may have the following layouts:
> > >>>>>>>>>>>
> > >>>>>>>>>>> 1. Ka -> Vbc
> > >>>>>>>>>>> 2. Kab -> Vc
> > >>>>>>>>>>> 3. Kabc -> Boolean.TRUE
> > >>>>>>>>>>>
> > >>>>>>>>>>> The only 1 is a correct layout, others are plain wrong
> > >>>> variants
> > >>>>>> (but
> > >>>>>>>>> they
> > >>>>>>>>>>> are still possible with Resolvers) because everything that
> > >>>> does
> > >>>>>> not
> > >>>>>>>>> make
> > >>>>>>>>>>> Key unique must be in Value.
> > >>>>>>>>>>>
> > >>>>>>>>>>> We want to clearly state that if you have something in Key,
> > >>>> that
> > >>>>>> is
> > >>>>>>>> not
> > >>>>>>>>>>> part of equals(), then the Key is invalid and that stuff must
> > >>>> be
> > >>>>>> in
> > >>>>>>>>>> Value.
> > >>>>>>>>>>> This allows us to rely on binary representation of a Key to
> be
> > >>>>>>>> stable
> > >>>>>>>>> and
> > >>>>>>>>>>> have some more optimizations and code simplifications with
> > >>>>> respect
> > >>>>>>>> to
> > >>>>>>>>>> these
> > >>>>>>>>>>> assumptions.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Sergi
> > >>>>>>>>>>>
> > >>>>>>>>>>> 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
> > >>>>>>>>>>> valentin.kulichenko@gmail.com>:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Even with my vast expirience I would never claim that I've
> > >>>>> seen
> > >>>>>>>>>>>> "everything" :)
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> What do you mean by stable binary key representation and how
> > >>>>>>>>> resolvers
> > >>>>>>>>>>> make
> > >>>>>>>>>>>> it unstable?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> -Val
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
> > >>>>>>>>>> sergi.vladykin@gmail.com
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Val,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I know that you have really vast experience in Ignite
> > >>>>>>>> deployments
> > >>>>>>>>> and
> > >>>>>>>>>>>>> probably saw everything that can happen. Did you ever see
> > >>>>>>>> identity
> > >>>>>>>>>>>>> resolvers use in real life? I guess no.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Hibernate example is bad here, because if their key is
> > >>>>>> unstable
> > >>>>>>>>>> across
> > >>>>>>>>>>>>> multiple JVMs, it means that it was not designed for
> > >>>>>> distributed
> > >>>>>>>>>>> caches a
> > >>>>>>>>>>>>> priori.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Also knowing in advance about stable binary key
> > >>>>> representation
> > >>>>>>>>> allows
> > >>>>>>>>>>> us
> > >>>>>>>>>>>> to
> > >>>>>>>>>>>>> apply additional optimizations, like comparing keys
> > >>>> without
> > >>>>>>>>> detaching
> > >>>>>>>>>>>> them
> > >>>>>>>>>>>>> from offheap memory.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> We always will be able to add this stuff back if we see
> > >>>>> users
> > >>>>>>>>> really
> > >>>>>>>>>>> need
> > >>>>>>>>>>>>> it. Let's remove it for 2.0.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Sergi
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
> > >>>>>>>>>>>>> valentin.kulichenko@gmail.com>:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Alex,
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> To be honest, I don't understand the reasoning behind
> > >>>> the
> > >>>>>>>>> removal.
> > >>>>>>>>>> I
> > >>>>>>>>>>>>> think
> > >>>>>>>>>>>>>> resolvers provide good flexibility for different corner
> > >>>>>> cases
> > >>>>>>>> and
> > >>>>>>>>>>> it's
> > >>>>>>>>>>>> a
> > >>>>>>>>>>>>>> good thing to have them. Note that they can be applied
> > >>>> not
> > >>>>>>>> only
> > >>>>>>>>> to
> > >>>>>>>>>>>> cache
> > >>>>>>>>>>>>>> keys, but to any binary objects.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Hibernate issue is actually a good example of such use
> > >>>>> case.
> > >>>>>>>> The
> > >>>>>>>>>> fact
> > >>>>>>>>>>>>> that
> > >>>>>>>>>>>>>> we found an alternative solution doesn't actually mean
> > >>>>>>>> anything,
> > >>>>>>>>>>>> because
> > >>>>>>>>>>>>>> what if this happened not in our module, but in user's
> > >>>>>>>>> application?
> > >>>>>>>>>>>>>> Unfortunately, we can't predict everything.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Error proneness is not a very strong argument either,
> > >>>>>> because
> > >>>>>>>> in
> > >>>>>>>>> my
> > >>>>>>>>>>>> view
> > >>>>>>>>>>>>>> these resolvers are as much error prone as
> > >>>> BinaryIdMapper,
> > >>>>>> for
> > >>>>>>>>>>> example.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> -Val
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
> > >>>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Denis,
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Can you suggest a use-case where identity resolver is
> > >>>>>> needed
> > >>>>>>>>>> (given
> > >>>>>>>>>>>>> that
> > >>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>> agree that a key must contain only valuable fields)?
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> 2017-04-05 22:08 GMT+03:00 Denis Magda <
> > >>>>> dmagda@apache.org
> > >>>>>>> :
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Where do you want to remove the identity resolvers
> > >>>>> from?
> > >>>>>>>> If
> > >>>>>>>>>> it’s
> > >>>>>>>>>>>>>> related
> > >>>>>>>>>>>>>>>> to the internals of Hibernate module then it’s fine
> > >>>>> but
> > >>>>>> if
> > >>>>>>>>> you
> > >>>>>>>>>>>>> suggest
> > >>>>>>>>>>>>>>>> removing identity resolvers public interfaces then
> > >>>> it
> > >>>>>>>> might
> > >>>>>>>>> be
> > >>>>>>>>>> a
> > >>>>>>>>>>>>> haste
> > >>>>>>>>>>>>>>>> decision.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> —
> > >>>>>>>>>>>>>>>> Denis
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
> > >>>>>>>>>>>>>>> alexey.goncharuk@gmail.com>
> > >>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> +1, I see no other reasons to keep it.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
> > >>>>>>>>>>>>> sergi.vladykin@gmail.com
> > >>>>>>>>>>>>>>> :
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> +1
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Lets drop them.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Sergi
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
> > >>>>>>>>>>>>>>>>>> dmitriy.govorukhin@gmail.com>
> > >>>>>>>>>>>>>>>>>> :
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Hi guys, i implemented proxy for IgniteCache in
> > >>>>>>>> hibernate
> > >>>>>>>>>>>>>>> integration,
> > >>>>>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>> proxy transformate cacheKey to our key wrapper,
> > >>>>>> leaves
> > >>>>>>>>> only
> > >>>>>>>>>>>>>> required
> > >>>>>>>>>>>>>>>>>>> field. I think we can remove identity resolve,
> > >>>> it
> > >>>>>>>> should
> > >>>>>>>>>> not
> > >>>>>>>>>>>>> broke
> > >>>>>>>>>>>>>>>>>>> integration with hibernate. Any objections?
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin
> > >>>>>> Kulichenko
> > >>>>>>>> <
> > >>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> I'm not saying there is no alternative
> > >>>> solution.
> > >>>>>> But
> > >>>>>>>>> let's
> > >>>>>>>>>>>>>> implement
> > >>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>> prove that it works first, and remove resolvers
> > >>>>>> only
> > >>>>>>>>> after
> > >>>>>>>>>>>> that.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> -Val
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi
> > >>>> Vladykin
> > >>>>> <
> > >>>>>>>>>>>>>>>>>>> sergi.vladykin@gmail.com
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Guys, nothing is impossible if you know a bit
> > >>>>>> about
> > >>>>>>>>>>>> reflection
> > >>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>> Java
> > >>>>>>>>>>>>>>>>>>> :)
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> We had a look at the CacheKey class and it is
> > >>>>>> easily
> > >>>>>>>>>>>>> replaceable.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Sergi
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
> > >>>>>>>>>>>>>>> dsetrakyan@apache.org
> > >>>>>>>>>>>>>>>>>>> :
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin
> > >>>>>>>> Kulichenko
> > >>>>>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> "Hibernate key" is the CacheKey class I was
> > >>>>>>>> referring
> > >>>>>>>>>> to.
> > >>>>>>>>>>>>> It's
> > >>>>>>>>>>>>>>>>>>>> provided
> > >>>>>>>>>>>>>>>>>>>>>> by
> > >>>>>>>>>>>>>>>>>>>>>>> Hibernate, not by user and not by us. So I'm
> > >>>>> not
> > >>>>>>>> sure
> > >>>>>>>>>>> it's
> > >>>>>>>>>>>>>>>>>> possible
> > >>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>> replace it.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> If it is impossible to replace or get rid of
> > >>>>> the
> > >>>>>>>>>> Hibernate
> > >>>>>>>>>>>>> key,
> > >>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>> this
> > >>>>>>>>>>>>>>>>>>>>>> discussion valid at all?
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>
> > >>
> >
> >
>
Re: Stable binary key representation
Posted by Igor Sapego <is...@apache.org>.
Denis,
The whole binary representation of the object is used now
for hash code generation and equality comparison. So the
answer - all fields are used for this.
Best Regards,
Igor
On Mon, Apr 10, 2017 at 9:50 PM, Denis Magda <dm...@apache.org> wrote:
> Considering this simple example
>
> INSERT (id, orgId, name, age, address) into Person…
>
> where id and orgId define Person’s affinity key - PersonKey(id, orgId)
>
> How do we know which fields to use for hash code generation and equality
> comparison? QueryEntity?
>
> No, it’s unclear how to document it properly.
>
> —
> Denis
>
> > On Apr 10, 2017, at 11:14 AM, Vladimir Ozerov <vo...@gridgain.com>
> wrote:
> >
> > There is no more such resolver. It was removed.
> >
> > On Mon, Apr 10, 2017 at 8:58 PM, Denis Magda <dm...@apache.org> wrote:
> >
> >> Vovan,
> >>
> >> Before I fix the documentation, what’t the replacement for
> >> BinaryFieldIdentiyResolver we used to define field for hash code
> >> calculation and equality comparison when DML statements are used?
> >> https://apacheignite.readme.io/docs/binary-marshaller#
> >> section-binary-field-identity-resolver <https://apacheignite.readme.
> >> io/docs/binary-marshaller#section-binary-field-identity-resolver>
> >>
> >> —
> >> Denis
> >>
> >>> On Apr 9, 2017, at 7:39 AM, Vladimir Ozerov <vo...@gridgain.com>
> >> wrote:
> >>>
> >>> Resolvers were essential for DML because we had broken comparison
> >> semantics
> >>> of binary objects. This is not the case now.
> >>>
> >>> Resolver as a whole is normal practice. E.g. it is implemented in .NET
> on
> >>> core language level and widely used in many cases. Hazelcast has it as
> >> well
> >>> AFAIK. So it is wrong to think that the whole idea is useless. Think of
> >> it
> >>> as a comparator's brother.
> >>>
> >>> The only reason why we need to remove it is missing hash index in new
> >>> architecture. It makes sense, as it is better to have AI 2.0 without
> >> them,
> >>> than no AI 2.0 :-)
> >>>
> >>> 09 апр. 2017 г. 17:31 пользователь "Sergi Vladykin" <
> >>> sergi.vladykin@gmail.com> написал:
> >>>
> >>>> I guess Resolvers were added to DML just because they already existed
> >> since
> >>>> 1.9 and we were forced to support them in all the parts of our
> product.
> >>>>
> >>>> We have to stop this practice to add features without clear real life
> >> use
> >>>> cases for them.
> >>>>
> >>>> Sergi
> >>>>
> >>>> 2017-04-09 17:00 GMT+03:00 Denis Magda <dm...@gridgain.com>:
> >>>>
> >>>>> Sergi, Vovan,
> >>>>>
> >>>>> Sorry for being annoying but I still didn't get an answer on whether
> >> the
> >>>>> resolvers are the must for DML. The main reason why we made them up
> >> some
> >>>>> time ago is to support specific DML use cases. However I can't recall
> >> the
> >>>>> use cases.
> >>>>>
> >>>>> --
> >>>>> Denis
> >>>>>
> >>>>> On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <
> >> sergi.vladykin@gmail.com
> >>>>>
> >>>>> wrote:
> >>>>>
> >>>>>> Ok, we need to do 2 things here:
> >>>>>>
> >>>>>> 1. Drop the resolvers from the source code.
> >>>>>> 2. Write a good page in docs on "What makes a correct cache key".
> >>>>>>
> >>>>>> Who can do that?
> >>>>>>
> >>>>>> Sergi
> >>>>>>
> >>>>>> 2017-04-07 9:48 GMT+03:00 Sergi Vladykin <sergi.vladykin@gmail.com
> >:
> >>>>>>
> >>>>>>> It is possible to try adding support of comparison to Resolvers,
> but
> >>>>> the
> >>>>>>> whole approach looks wrong and for now it is better to get rid of
> it
> >>>>>> while
> >>>>>>> we have a chance to break compatibility.
> >>>>>>>
> >>>>>>> Sergi
> >>>>>>>
> >>>>>>> 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
> >>>>>>> valentin.kulichenko@gmail.com>:
> >>>>>>>
> >>>>>>>> The discussion should've been started with that :) If supporting
> >>>>>> resolvers
> >>>>>>>> in new architecture is not possible or means too big effort, then
> >>>> it's
> >>>>>>>> definitely not worth it.
> >>>>>>>>
> >>>>>>>> -Val
> >>>>>>>>
> >>>>>>>> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <
> >>>> vozerov@gridgain.com
> >>>>>>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Dima,
> >>>>>>>>>
> >>>>>>>>> Yes, they may explode some internals of our indexes.
> >>>>>>>>>
> >>>>>>>>> 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan" <
> >>>>>>>>> dsetrakyan@apache.org> написал:
> >>>>>>>>>
> >>>>>>>>>> Guys,
> >>>>>>>>>>
> >>>>>>>>>> Isn't the main issue here that we cannot use the Identity
> >>>>> Resolvers
> >>>>>> in
> >>>>>>>>>> BTrees in the 2.0 version? If yes, then we have to remove them
> >>>> no
> >>>>>>>> matter
> >>>>>>>>>> what.
> >>>>>>>>>>
> >>>>>>>>>> D.
> >>>>>>>>>>
> >>>>>>>>>> On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
> >>>>>>>> sergi.vladykin@gmail.com
> >>>>>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Binary key representation is stable when we always have equal
> >>>>>>>>> serialized
> >>>>>>>>>>> bytes when the original keys are equal.
> >>>>>>>>>>>
> >>>>>>>>>>> Resolver allows you to have some extra info in the Key and
> >>>> equal
> >>>>>>>> Keys
> >>>>>>>>>> will
> >>>>>>>>>>> be serialized into different bytes, which is wrong.
> >>>>>>>>>>>
> >>>>>>>>>>> Look at the example what you can do with resolvers:
> >>>>>>>>>>>
> >>>>>>>>>>> We may have some data entry with fields a, b, c. Let's say the
> >>>>>>>> unique
> >>>>>>>>>> part
> >>>>>>>>>>> here is `a` and it the only fields used in Key equals() and
> >>>>>>>> hashCode().
> >>>>>>>>>>> Still we may have the following layouts:
> >>>>>>>>>>>
> >>>>>>>>>>> 1. Ka -> Vbc
> >>>>>>>>>>> 2. Kab -> Vc
> >>>>>>>>>>> 3. Kabc -> Boolean.TRUE
> >>>>>>>>>>>
> >>>>>>>>>>> The only 1 is a correct layout, others are plain wrong
> >>>> variants
> >>>>>> (but
> >>>>>>>>> they
> >>>>>>>>>>> are still possible with Resolvers) because everything that
> >>>> does
> >>>>>> not
> >>>>>>>>> make
> >>>>>>>>>>> Key unique must be in Value.
> >>>>>>>>>>>
> >>>>>>>>>>> We want to clearly state that if you have something in Key,
> >>>> that
> >>>>>> is
> >>>>>>>> not
> >>>>>>>>>>> part of equals(), then the Key is invalid and that stuff must
> >>>> be
> >>>>>> in
> >>>>>>>>>> Value.
> >>>>>>>>>>> This allows us to rely on binary representation of a Key to be
> >>>>>>>> stable
> >>>>>>>>> and
> >>>>>>>>>>> have some more optimizations and code simplifications with
> >>>>> respect
> >>>>>>>> to
> >>>>>>>>>> these
> >>>>>>>>>>> assumptions.
> >>>>>>>>>>>
> >>>>>>>>>>> Sergi
> >>>>>>>>>>>
> >>>>>>>>>>> 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
> >>>>>>>>>>> valentin.kulichenko@gmail.com>:
> >>>>>>>>>>>
> >>>>>>>>>>>> Even with my vast expirience I would never claim that I've
> >>>>> seen
> >>>>>>>>>>>> "everything" :)
> >>>>>>>>>>>>
> >>>>>>>>>>>> What do you mean by stable binary key representation and how
> >>>>>>>>> resolvers
> >>>>>>>>>>> make
> >>>>>>>>>>>> it unstable?
> >>>>>>>>>>>>
> >>>>>>>>>>>> -Val
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
> >>>>>>>>>> sergi.vladykin@gmail.com
> >>>>>>>>>>>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Val,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I know that you have really vast experience in Ignite
> >>>>>>>> deployments
> >>>>>>>>> and
> >>>>>>>>>>>>> probably saw everything that can happen. Did you ever see
> >>>>>>>> identity
> >>>>>>>>>>>>> resolvers use in real life? I guess no.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hibernate example is bad here, because if their key is
> >>>>>> unstable
> >>>>>>>>>> across
> >>>>>>>>>>>>> multiple JVMs, it means that it was not designed for
> >>>>>> distributed
> >>>>>>>>>>> caches a
> >>>>>>>>>>>>> priori.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Also knowing in advance about stable binary key
> >>>>> representation
> >>>>>>>>> allows
> >>>>>>>>>>> us
> >>>>>>>>>>>> to
> >>>>>>>>>>>>> apply additional optimizations, like comparing keys
> >>>> without
> >>>>>>>>> detaching
> >>>>>>>>>>>> them
> >>>>>>>>>>>>> from offheap memory.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> We always will be able to add this stuff back if we see
> >>>>> users
> >>>>>>>>> really
> >>>>>>>>>>> need
> >>>>>>>>>>>>> it. Let's remove it for 2.0.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Sergi
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
> >>>>>>>>>>>>> valentin.kulichenko@gmail.com>:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Alex,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> To be honest, I don't understand the reasoning behind
> >>>> the
> >>>>>>>>> removal.
> >>>>>>>>>> I
> >>>>>>>>>>>>> think
> >>>>>>>>>>>>>> resolvers provide good flexibility for different corner
> >>>>>> cases
> >>>>>>>> and
> >>>>>>>>>>> it's
> >>>>>>>>>>>> a
> >>>>>>>>>>>>>> good thing to have them. Note that they can be applied
> >>>> not
> >>>>>>>> only
> >>>>>>>>> to
> >>>>>>>>>>>> cache
> >>>>>>>>>>>>>> keys, but to any binary objects.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hibernate issue is actually a good example of such use
> >>>>> case.
> >>>>>>>> The
> >>>>>>>>>> fact
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>> we found an alternative solution doesn't actually mean
> >>>>>>>> anything,
> >>>>>>>>>>>> because
> >>>>>>>>>>>>>> what if this happened not in our module, but in user's
> >>>>>>>>> application?
> >>>>>>>>>>>>>> Unfortunately, we can't predict everything.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Error proneness is not a very strong argument either,
> >>>>>> because
> >>>>>>>> in
> >>>>>>>>> my
> >>>>>>>>>>>> view
> >>>>>>>>>>>>>> these resolvers are as much error prone as
> >>>> BinaryIdMapper,
> >>>>>> for
> >>>>>>>>>>> example.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> -Val
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
> >>>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Denis,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Can you suggest a use-case where identity resolver is
> >>>>>> needed
> >>>>>>>>>> (given
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>> agree that a key must contain only valuable fields)?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 2017-04-05 22:08 GMT+03:00 Denis Magda <
> >>>>> dmagda@apache.org
> >>>>>>> :
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Where do you want to remove the identity resolvers
> >>>>> from?
> >>>>>>>> If
> >>>>>>>>>> it’s
> >>>>>>>>>>>>>> related
> >>>>>>>>>>>>>>>> to the internals of Hibernate module then it’s fine
> >>>>> but
> >>>>>> if
> >>>>>>>>> you
> >>>>>>>>>>>>> suggest
> >>>>>>>>>>>>>>>> removing identity resolvers public interfaces then
> >>>> it
> >>>>>>>> might
> >>>>>>>>> be
> >>>>>>>>>> a
> >>>>>>>>>>>>> haste
> >>>>>>>>>>>>>>>> decision.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> —
> >>>>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
> >>>>>>>>>>>>>>> alexey.goncharuk@gmail.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> +1, I see no other reasons to keep it.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
> >>>>>>>>>>>>> sergi.vladykin@gmail.com
> >>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> +1
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Lets drop them.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Sergi
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
> >>>>>>>>>>>>>>>>>> dmitriy.govorukhin@gmail.com>
> >>>>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hi guys, i implemented proxy for IgniteCache in
> >>>>>>>> hibernate
> >>>>>>>>>>>>>>> integration,
> >>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>> proxy transformate cacheKey to our key wrapper,
> >>>>>> leaves
> >>>>>>>>> only
> >>>>>>>>>>>>>> required
> >>>>>>>>>>>>>>>>>>> field. I think we can remove identity resolve,
> >>>> it
> >>>>>>>> should
> >>>>>>>>>> not
> >>>>>>>>>>>>> broke
> >>>>>>>>>>>>>>>>>>> integration with hibernate. Any objections?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin
> >>>>>> Kulichenko
> >>>>>>>> <
> >>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I'm not saying there is no alternative
> >>>> solution.
> >>>>>> But
> >>>>>>>>> let's
> >>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>> prove that it works first, and remove resolvers
> >>>>>> only
> >>>>>>>>> after
> >>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> -Val
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi
> >>>> Vladykin
> >>>>> <
> >>>>>>>>>>>>>>>>>>> sergi.vladykin@gmail.com
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Guys, nothing is impossible if you know a bit
> >>>>>> about
> >>>>>>>>>>>> reflection
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>> Java
> >>>>>>>>>>>>>>>>>>> :)
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> We had a look at the CacheKey class and it is
> >>>>>> easily
> >>>>>>>>>>>>> replaceable.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Sergi
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
> >>>>>>>>>>>>>>> dsetrakyan@apache.org
> >>>>>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin
> >>>>>>>> Kulichenko
> >>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> "Hibernate key" is the CacheKey class I was
> >>>>>>>> referring
> >>>>>>>>>> to.
> >>>>>>>>>>>>> It's
> >>>>>>>>>>>>>>>>>>>> provided
> >>>>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>>> Hibernate, not by user and not by us. So I'm
> >>>>> not
> >>>>>>>> sure
> >>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>> possible
> >>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>> replace it.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> If it is impossible to replace or get rid of
> >>>>> the
> >>>>>>>>>> Hibernate
> >>>>>>>>>>>>> key,
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>> discussion valid at all?
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> >>
>
>
Re: Stable binary key representation
Posted by Denis Magda <dm...@apache.org>.
Considering this simple example
INSERT (id, orgId, name, age, address) into Person…
where id and orgId define Person’s affinity key - PersonKey(id, orgId)
How do we know which fields to use for hash code generation and equality comparison? QueryEntity?
No, it’s unclear how to document it properly.
—
Denis
> On Apr 10, 2017, at 11:14 AM, Vladimir Ozerov <vo...@gridgain.com> wrote:
>
> There is no more such resolver. It was removed.
>
> On Mon, Apr 10, 2017 at 8:58 PM, Denis Magda <dm...@apache.org> wrote:
>
>> Vovan,
>>
>> Before I fix the documentation, what’t the replacement for
>> BinaryFieldIdentiyResolver we used to define field for hash code
>> calculation and equality comparison when DML statements are used?
>> https://apacheignite.readme.io/docs/binary-marshaller#
>> section-binary-field-identity-resolver <https://apacheignite.readme.
>> io/docs/binary-marshaller#section-binary-field-identity-resolver>
>>
>> —
>> Denis
>>
>>> On Apr 9, 2017, at 7:39 AM, Vladimir Ozerov <vo...@gridgain.com>
>> wrote:
>>>
>>> Resolvers were essential for DML because we had broken comparison
>> semantics
>>> of binary objects. This is not the case now.
>>>
>>> Resolver as a whole is normal practice. E.g. it is implemented in .NET on
>>> core language level and widely used in many cases. Hazelcast has it as
>> well
>>> AFAIK. So it is wrong to think that the whole idea is useless. Think of
>> it
>>> as a comparator's brother.
>>>
>>> The only reason why we need to remove it is missing hash index in new
>>> architecture. It makes sense, as it is better to have AI 2.0 without
>> them,
>>> than no AI 2.0 :-)
>>>
>>> 09 апр. 2017 г. 17:31 пользователь "Sergi Vladykin" <
>>> sergi.vladykin@gmail.com> написал:
>>>
>>>> I guess Resolvers were added to DML just because they already existed
>> since
>>>> 1.9 and we were forced to support them in all the parts of our product.
>>>>
>>>> We have to stop this practice to add features without clear real life
>> use
>>>> cases for them.
>>>>
>>>> Sergi
>>>>
>>>> 2017-04-09 17:00 GMT+03:00 Denis Magda <dm...@gridgain.com>:
>>>>
>>>>> Sergi, Vovan,
>>>>>
>>>>> Sorry for being annoying but I still didn't get an answer on whether
>> the
>>>>> resolvers are the must for DML. The main reason why we made them up
>> some
>>>>> time ago is to support specific DML use cases. However I can't recall
>> the
>>>>> use cases.
>>>>>
>>>>> --
>>>>> Denis
>>>>>
>>>>> On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <
>> sergi.vladykin@gmail.com
>>>>>
>>>>> wrote:
>>>>>
>>>>>> Ok, we need to do 2 things here:
>>>>>>
>>>>>> 1. Drop the resolvers from the source code.
>>>>>> 2. Write a good page in docs on "What makes a correct cache key".
>>>>>>
>>>>>> Who can do that?
>>>>>>
>>>>>> Sergi
>>>>>>
>>>>>> 2017-04-07 9:48 GMT+03:00 Sergi Vladykin <se...@gmail.com>:
>>>>>>
>>>>>>> It is possible to try adding support of comparison to Resolvers, but
>>>>> the
>>>>>>> whole approach looks wrong and for now it is better to get rid of it
>>>>>> while
>>>>>>> we have a chance to break compatibility.
>>>>>>>
>>>>>>> Sergi
>>>>>>>
>>>>>>> 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
>>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>>
>>>>>>>> The discussion should've been started with that :) If supporting
>>>>>> resolvers
>>>>>>>> in new architecture is not possible or means too big effort, then
>>>> it's
>>>>>>>> definitely not worth it.
>>>>>>>>
>>>>>>>> -Val
>>>>>>>>
>>>>>>>> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <
>>>> vozerov@gridgain.com
>>>>>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Dima,
>>>>>>>>>
>>>>>>>>> Yes, they may explode some internals of our indexes.
>>>>>>>>>
>>>>>>>>> 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan" <
>>>>>>>>> dsetrakyan@apache.org> написал:
>>>>>>>>>
>>>>>>>>>> Guys,
>>>>>>>>>>
>>>>>>>>>> Isn't the main issue here that we cannot use the Identity
>>>>> Resolvers
>>>>>> in
>>>>>>>>>> BTrees in the 2.0 version? If yes, then we have to remove them
>>>> no
>>>>>>>> matter
>>>>>>>>>> what.
>>>>>>>>>>
>>>>>>>>>> D.
>>>>>>>>>>
>>>>>>>>>> On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Binary key representation is stable when we always have equal
>>>>>>>>> serialized
>>>>>>>>>>> bytes when the original keys are equal.
>>>>>>>>>>>
>>>>>>>>>>> Resolver allows you to have some extra info in the Key and
>>>> equal
>>>>>>>> Keys
>>>>>>>>>> will
>>>>>>>>>>> be serialized into different bytes, which is wrong.
>>>>>>>>>>>
>>>>>>>>>>> Look at the example what you can do with resolvers:
>>>>>>>>>>>
>>>>>>>>>>> We may have some data entry with fields a, b, c. Let's say the
>>>>>>>> unique
>>>>>>>>>> part
>>>>>>>>>>> here is `a` and it the only fields used in Key equals() and
>>>>>>>> hashCode().
>>>>>>>>>>> Still we may have the following layouts:
>>>>>>>>>>>
>>>>>>>>>>> 1. Ka -> Vbc
>>>>>>>>>>> 2. Kab -> Vc
>>>>>>>>>>> 3. Kabc -> Boolean.TRUE
>>>>>>>>>>>
>>>>>>>>>>> The only 1 is a correct layout, others are plain wrong
>>>> variants
>>>>>> (but
>>>>>>>>> they
>>>>>>>>>>> are still possible with Resolvers) because everything that
>>>> does
>>>>>> not
>>>>>>>>> make
>>>>>>>>>>> Key unique must be in Value.
>>>>>>>>>>>
>>>>>>>>>>> We want to clearly state that if you have something in Key,
>>>> that
>>>>>> is
>>>>>>>> not
>>>>>>>>>>> part of equals(), then the Key is invalid and that stuff must
>>>> be
>>>>>> in
>>>>>>>>>> Value.
>>>>>>>>>>> This allows us to rely on binary representation of a Key to be
>>>>>>>> stable
>>>>>>>>> and
>>>>>>>>>>> have some more optimizations and code simplifications with
>>>>> respect
>>>>>>>> to
>>>>>>>>>> these
>>>>>>>>>>> assumptions.
>>>>>>>>>>>
>>>>>>>>>>> Sergi
>>>>>>>>>>>
>>>>>>>>>>> 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
>>>>>>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>>>>>>
>>>>>>>>>>>> Even with my vast expirience I would never claim that I've
>>>>> seen
>>>>>>>>>>>> "everything" :)
>>>>>>>>>>>>
>>>>>>>>>>>> What do you mean by stable binary key representation and how
>>>>>>>>> resolvers
>>>>>>>>>>> make
>>>>>>>>>>>> it unstable?
>>>>>>>>>>>>
>>>>>>>>>>>> -Val
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
>>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Val,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I know that you have really vast experience in Ignite
>>>>>>>> deployments
>>>>>>>>> and
>>>>>>>>>>>>> probably saw everything that can happen. Did you ever see
>>>>>>>> identity
>>>>>>>>>>>>> resolvers use in real life? I guess no.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hibernate example is bad here, because if their key is
>>>>>> unstable
>>>>>>>>>> across
>>>>>>>>>>>>> multiple JVMs, it means that it was not designed for
>>>>>> distributed
>>>>>>>>>>> caches a
>>>>>>>>>>>>> priori.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Also knowing in advance about stable binary key
>>>>> representation
>>>>>>>>> allows
>>>>>>>>>>> us
>>>>>>>>>>>> to
>>>>>>>>>>>>> apply additional optimizations, like comparing keys
>>>> without
>>>>>>>>> detaching
>>>>>>>>>>>> them
>>>>>>>>>>>>> from offheap memory.
>>>>>>>>>>>>>
>>>>>>>>>>>>> We always will be able to add this stuff back if we see
>>>>> users
>>>>>>>>> really
>>>>>>>>>>> need
>>>>>>>>>>>>> it. Let's remove it for 2.0.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
>>>>>>>>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Alex,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To be honest, I don't understand the reasoning behind
>>>> the
>>>>>>>>> removal.
>>>>>>>>>> I
>>>>>>>>>>>>> think
>>>>>>>>>>>>>> resolvers provide good flexibility for different corner
>>>>>> cases
>>>>>>>> and
>>>>>>>>>>> it's
>>>>>>>>>>>> a
>>>>>>>>>>>>>> good thing to have them. Note that they can be applied
>>>> not
>>>>>>>> only
>>>>>>>>> to
>>>>>>>>>>>> cache
>>>>>>>>>>>>>> keys, but to any binary objects.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hibernate issue is actually a good example of such use
>>>>> case.
>>>>>>>> The
>>>>>>>>>> fact
>>>>>>>>>>>>> that
>>>>>>>>>>>>>> we found an alternative solution doesn't actually mean
>>>>>>>> anything,
>>>>>>>>>>>> because
>>>>>>>>>>>>>> what if this happened not in our module, but in user's
>>>>>>>>> application?
>>>>>>>>>>>>>> Unfortunately, we can't predict everything.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Error proneness is not a very strong argument either,
>>>>>> because
>>>>>>>> in
>>>>>>>>> my
>>>>>>>>>>>> view
>>>>>>>>>>>>>> these resolvers are as much error prone as
>>>> BinaryIdMapper,
>>>>>> for
>>>>>>>>>>> example.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -Val
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
>>>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Denis,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Can you suggest a use-case where identity resolver is
>>>>>> needed
>>>>>>>>>> (given
>>>>>>>>>>>>> that
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>> agree that a key must contain only valuable fields)?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2017-04-05 22:08 GMT+03:00 Denis Magda <
>>>>> dmagda@apache.org
>>>>>>> :
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Where do you want to remove the identity resolvers
>>>>> from?
>>>>>>>> If
>>>>>>>>>> it’s
>>>>>>>>>>>>>> related
>>>>>>>>>>>>>>>> to the internals of Hibernate module then it’s fine
>>>>> but
>>>>>> if
>>>>>>>>> you
>>>>>>>>>>>>> suggest
>>>>>>>>>>>>>>>> removing identity resolvers public interfaces then
>>>> it
>>>>>>>> might
>>>>>>>>> be
>>>>>>>>>> a
>>>>>>>>>>>>> haste
>>>>>>>>>>>>>>>> decision.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> +1, I see no other reasons to keep it.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
>>>>>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Lets drop them.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
>>>>>>>>>>>>>>>>>> dmitriy.govorukhin@gmail.com>
>>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi guys, i implemented proxy for IgniteCache in
>>>>>>>> hibernate
>>>>>>>>>>>>>>> integration,
>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>> proxy transformate cacheKey to our key wrapper,
>>>>>> leaves
>>>>>>>>> only
>>>>>>>>>>>>>> required
>>>>>>>>>>>>>>>>>>> field. I think we can remove identity resolve,
>>>> it
>>>>>>>> should
>>>>>>>>>> not
>>>>>>>>>>>>> broke
>>>>>>>>>>>>>>>>>>> integration with hibernate. Any objections?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin
>>>>>> Kulichenko
>>>>>>>> <
>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I'm not saying there is no alternative
>>>> solution.
>>>>>> But
>>>>>>>>> let's
>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> prove that it works first, and remove resolvers
>>>>>> only
>>>>>>>>> after
>>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> -Val
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi
>>>> Vladykin
>>>>> <
>>>>>>>>>>>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Guys, nothing is impossible if you know a bit
>>>>>> about
>>>>>>>>>>>> reflection
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> Java
>>>>>>>>>>>>>>>>>>> :)
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> We had a look at the CacheKey class and it is
>>>>>> easily
>>>>>>>>>>>>> replaceable.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
>>>>>>>>>>>>>>> dsetrakyan@apache.org
>>>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin
>>>>>>>> Kulichenko
>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> "Hibernate key" is the CacheKey class I was
>>>>>>>> referring
>>>>>>>>>> to.
>>>>>>>>>>>>> It's
>>>>>>>>>>>>>>>>>>>> provided
>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>> Hibernate, not by user and not by us. So I'm
>>>>> not
>>>>>>>> sure
>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>> possible
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> replace it.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> If it is impossible to replace or get rid of
>>>>> the
>>>>>>>>>> Hibernate
>>>>>>>>>>>>> key,
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>> discussion valid at all?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>
>>
Re: Stable binary key representation
Posted by Vladimir Ozerov <vo...@gridgain.com>.
There is no more such resolver. It was removed.
On Mon, Apr 10, 2017 at 8:58 PM, Denis Magda <dm...@apache.org> wrote:
> Vovan,
>
> Before I fix the documentation, what’t the replacement for
> BinaryFieldIdentiyResolver we used to define field for hash code
> calculation and equality comparison when DML statements are used?
> https://apacheignite.readme.io/docs/binary-marshaller#
> section-binary-field-identity-resolver <https://apacheignite.readme.
> io/docs/binary-marshaller#section-binary-field-identity-resolver>
>
> —
> Denis
>
> > On Apr 9, 2017, at 7:39 AM, Vladimir Ozerov <vo...@gridgain.com>
> wrote:
> >
> > Resolvers were essential for DML because we had broken comparison
> semantics
> > of binary objects. This is not the case now.
> >
> > Resolver as a whole is normal practice. E.g. it is implemented in .NET on
> > core language level and widely used in many cases. Hazelcast has it as
> well
> > AFAIK. So it is wrong to think that the whole idea is useless. Think of
> it
> > as a comparator's brother.
> >
> > The only reason why we need to remove it is missing hash index in new
> > architecture. It makes sense, as it is better to have AI 2.0 without
> them,
> > than no AI 2.0 :-)
> >
> > 09 апр. 2017 г. 17:31 пользователь "Sergi Vladykin" <
> > sergi.vladykin@gmail.com> написал:
> >
> >> I guess Resolvers were added to DML just because they already existed
> since
> >> 1.9 and we were forced to support them in all the parts of our product.
> >>
> >> We have to stop this practice to add features without clear real life
> use
> >> cases for them.
> >>
> >> Sergi
> >>
> >> 2017-04-09 17:00 GMT+03:00 Denis Magda <dm...@gridgain.com>:
> >>
> >>> Sergi, Vovan,
> >>>
> >>> Sorry for being annoying but I still didn't get an answer on whether
> the
> >>> resolvers are the must for DML. The main reason why we made them up
> some
> >>> time ago is to support specific DML use cases. However I can't recall
> the
> >>> use cases.
> >>>
> >>> --
> >>> Denis
> >>>
> >>> On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <
> sergi.vladykin@gmail.com
> >>>
> >>> wrote:
> >>>
> >>>> Ok, we need to do 2 things here:
> >>>>
> >>>> 1. Drop the resolvers from the source code.
> >>>> 2. Write a good page in docs on "What makes a correct cache key".
> >>>>
> >>>> Who can do that?
> >>>>
> >>>> Sergi
> >>>>
> >>>> 2017-04-07 9:48 GMT+03:00 Sergi Vladykin <se...@gmail.com>:
> >>>>
> >>>>> It is possible to try adding support of comparison to Resolvers, but
> >>> the
> >>>>> whole approach looks wrong and for now it is better to get rid of it
> >>>> while
> >>>>> we have a chance to break compatibility.
> >>>>>
> >>>>> Sergi
> >>>>>
> >>>>> 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
> >>>>> valentin.kulichenko@gmail.com>:
> >>>>>
> >>>>>> The discussion should've been started with that :) If supporting
> >>>> resolvers
> >>>>>> in new architecture is not possible or means too big effort, then
> >> it's
> >>>>>> definitely not worth it.
> >>>>>>
> >>>>>> -Val
> >>>>>>
> >>>>>> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <
> >> vozerov@gridgain.com
> >>>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Dima,
> >>>>>>>
> >>>>>>> Yes, they may explode some internals of our indexes.
> >>>>>>>
> >>>>>>> 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan" <
> >>>>>>> dsetrakyan@apache.org> написал:
> >>>>>>>
> >>>>>>>> Guys,
> >>>>>>>>
> >>>>>>>> Isn't the main issue here that we cannot use the Identity
> >>> Resolvers
> >>>> in
> >>>>>>>> BTrees in the 2.0 version? If yes, then we have to remove them
> >> no
> >>>>>> matter
> >>>>>>>> what.
> >>>>>>>>
> >>>>>>>> D.
> >>>>>>>>
> >>>>>>>> On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
> >>>>>> sergi.vladykin@gmail.com
> >>>>>>>>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Binary key representation is stable when we always have equal
> >>>>>>> serialized
> >>>>>>>>> bytes when the original keys are equal.
> >>>>>>>>>
> >>>>>>>>> Resolver allows you to have some extra info in the Key and
> >> equal
> >>>>>> Keys
> >>>>>>>> will
> >>>>>>>>> be serialized into different bytes, which is wrong.
> >>>>>>>>>
> >>>>>>>>> Look at the example what you can do with resolvers:
> >>>>>>>>>
> >>>>>>>>> We may have some data entry with fields a, b, c. Let's say the
> >>>>>> unique
> >>>>>>>> part
> >>>>>>>>> here is `a` and it the only fields used in Key equals() and
> >>>>>> hashCode().
> >>>>>>>>> Still we may have the following layouts:
> >>>>>>>>>
> >>>>>>>>> 1. Ka -> Vbc
> >>>>>>>>> 2. Kab -> Vc
> >>>>>>>>> 3. Kabc -> Boolean.TRUE
> >>>>>>>>>
> >>>>>>>>> The only 1 is a correct layout, others are plain wrong
> >> variants
> >>>> (but
> >>>>>>> they
> >>>>>>>>> are still possible with Resolvers) because everything that
> >> does
> >>>> not
> >>>>>>> make
> >>>>>>>>> Key unique must be in Value.
> >>>>>>>>>
> >>>>>>>>> We want to clearly state that if you have something in Key,
> >> that
> >>>> is
> >>>>>> not
> >>>>>>>>> part of equals(), then the Key is invalid and that stuff must
> >> be
> >>>> in
> >>>>>>>> Value.
> >>>>>>>>> This allows us to rely on binary representation of a Key to be
> >>>>>> stable
> >>>>>>> and
> >>>>>>>>> have some more optimizations and code simplifications with
> >>> respect
> >>>>>> to
> >>>>>>>> these
> >>>>>>>>> assumptions.
> >>>>>>>>>
> >>>>>>>>> Sergi
> >>>>>>>>>
> >>>>>>>>> 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
> >>>>>>>>> valentin.kulichenko@gmail.com>:
> >>>>>>>>>
> >>>>>>>>>> Even with my vast expirience I would never claim that I've
> >>> seen
> >>>>>>>>>> "everything" :)
> >>>>>>>>>>
> >>>>>>>>>> What do you mean by stable binary key representation and how
> >>>>>>> resolvers
> >>>>>>>>> make
> >>>>>>>>>> it unstable?
> >>>>>>>>>>
> >>>>>>>>>> -Val
> >>>>>>>>>>
> >>>>>>>>>> On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
> >>>>>>>> sergi.vladykin@gmail.com
> >>>>>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Val,
> >>>>>>>>>>>
> >>>>>>>>>>> I know that you have really vast experience in Ignite
> >>>>>> deployments
> >>>>>>> and
> >>>>>>>>>>> probably saw everything that can happen. Did you ever see
> >>>>>> identity
> >>>>>>>>>>> resolvers use in real life? I guess no.
> >>>>>>>>>>>
> >>>>>>>>>>> Hibernate example is bad here, because if their key is
> >>>> unstable
> >>>>>>>> across
> >>>>>>>>>>> multiple JVMs, it means that it was not designed for
> >>>> distributed
> >>>>>>>>> caches a
> >>>>>>>>>>> priori.
> >>>>>>>>>>>
> >>>>>>>>>>> Also knowing in advance about stable binary key
> >>> representation
> >>>>>>> allows
> >>>>>>>>> us
> >>>>>>>>>> to
> >>>>>>>>>>> apply additional optimizations, like comparing keys
> >> without
> >>>>>>> detaching
> >>>>>>>>>> them
> >>>>>>>>>>> from offheap memory.
> >>>>>>>>>>>
> >>>>>>>>>>> We always will be able to add this stuff back if we see
> >>> users
> >>>>>>> really
> >>>>>>>>> need
> >>>>>>>>>>> it. Let's remove it for 2.0.
> >>>>>>>>>>>
> >>>>>>>>>>> Sergi
> >>>>>>>>>>>
> >>>>>>>>>>> 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
> >>>>>>>>>>> valentin.kulichenko@gmail.com>:
> >>>>>>>>>>>
> >>>>>>>>>>>> Alex,
> >>>>>>>>>>>>
> >>>>>>>>>>>> To be honest, I don't understand the reasoning behind
> >> the
> >>>>>>> removal.
> >>>>>>>> I
> >>>>>>>>>>> think
> >>>>>>>>>>>> resolvers provide good flexibility for different corner
> >>>> cases
> >>>>>> and
> >>>>>>>>> it's
> >>>>>>>>>> a
> >>>>>>>>>>>> good thing to have them. Note that they can be applied
> >> not
> >>>>>> only
> >>>>>>> to
> >>>>>>>>>> cache
> >>>>>>>>>>>> keys, but to any binary objects.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Hibernate issue is actually a good example of such use
> >>> case.
> >>>>>> The
> >>>>>>>> fact
> >>>>>>>>>>> that
> >>>>>>>>>>>> we found an alternative solution doesn't actually mean
> >>>>>> anything,
> >>>>>>>>>> because
> >>>>>>>>>>>> what if this happened not in our module, but in user's
> >>>>>>> application?
> >>>>>>>>>>>> Unfortunately, we can't predict everything.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Error proneness is not a very strong argument either,
> >>>> because
> >>>>>> in
> >>>>>>> my
> >>>>>>>>>> view
> >>>>>>>>>>>> these resolvers are as much error prone as
> >> BinaryIdMapper,
> >>>> for
> >>>>>>>>> example.
> >>>>>>>>>>>>
> >>>>>>>>>>>> -Val
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
> >>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Denis,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Can you suggest a use-case where identity resolver is
> >>>> needed
> >>>>>>>> (given
> >>>>>>>>>>> that
> >>>>>>>>>>>> we
> >>>>>>>>>>>>> agree that a key must contain only valuable fields)?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 2017-04-05 22:08 GMT+03:00 Denis Magda <
> >>> dmagda@apache.org
> >>>>> :
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Where do you want to remove the identity resolvers
> >>> from?
> >>>>>> If
> >>>>>>>> it’s
> >>>>>>>>>>>> related
> >>>>>>>>>>>>>> to the internals of Hibernate module then it’s fine
> >>> but
> >>>> if
> >>>>>>> you
> >>>>>>>>>>> suggest
> >>>>>>>>>>>>>> removing identity resolvers public interfaces then
> >> it
> >>>>>> might
> >>>>>>> be
> >>>>>>>> a
> >>>>>>>>>>> haste
> >>>>>>>>>>>>>> decision.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> —
> >>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
> >>>>>>>>>>>>> alexey.goncharuk@gmail.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> +1, I see no other reasons to keep it.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
> >>>>>>>>>>> sergi.vladykin@gmail.com
> >>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> +1
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Lets drop them.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Sergi
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
> >>>>>>>>>>>>>>>> dmitriy.govorukhin@gmail.com>
> >>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hi guys, i implemented proxy for IgniteCache in
> >>>>>> hibernate
> >>>>>>>>>>>>> integration,
> >>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>> proxy transformate cacheKey to our key wrapper,
> >>>> leaves
> >>>>>>> only
> >>>>>>>>>>>> required
> >>>>>>>>>>>>>>>>> field. I think we can remove identity resolve,
> >> it
> >>>>>> should
> >>>>>>>> not
> >>>>>>>>>>> broke
> >>>>>>>>>>>>>>>>> integration with hibernate. Any objections?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin
> >>>> Kulichenko
> >>>>>> <
> >>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I'm not saying there is no alternative
> >> solution.
> >>>> But
> >>>>>>> let's
> >>>>>>>>>>>> implement
> >>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>> prove that it works first, and remove resolvers
> >>>> only
> >>>>>>> after
> >>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> -Val
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi
> >> Vladykin
> >>> <
> >>>>>>>>>>>>>>>>> sergi.vladykin@gmail.com
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Guys, nothing is impossible if you know a bit
> >>>> about
> >>>>>>>>>> reflection
> >>>>>>>>>>> in
> >>>>>>>>>>>>>>>> Java
> >>>>>>>>>>>>>>>>> :)
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> We had a look at the CacheKey class and it is
> >>>> easily
> >>>>>>>>>>> replaceable.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Sergi
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
> >>>>>>>>>>>>> dsetrakyan@apache.org
> >>>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin
> >>>>>> Kulichenko
> >>>>>>> <
> >>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> "Hibernate key" is the CacheKey class I was
> >>>>>> referring
> >>>>>>>> to.
> >>>>>>>>>>> It's
> >>>>>>>>>>>>>>>>>> provided
> >>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>> Hibernate, not by user and not by us. So I'm
> >>> not
> >>>>>> sure
> >>>>>>>>> it's
> >>>>>>>>>>>>>>>> possible
> >>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> replace it.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> If it is impossible to replace or get rid of
> >>> the
> >>>>>>>> Hibernate
> >>>>>>>>>>> key,
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>> discussion valid at all?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>
Re: Stable binary key representation
Posted by Denis Magda <dm...@apache.org>.
Vovan,
Before I fix the documentation, what’t the replacement for BinaryFieldIdentiyResolver we used to define field for hash code calculation and equality comparison when DML statements are used?
https://apacheignite.readme.io/docs/binary-marshaller#section-binary-field-identity-resolver <https://apacheignite.readme.io/docs/binary-marshaller#section-binary-field-identity-resolver>
—
Denis
> On Apr 9, 2017, at 7:39 AM, Vladimir Ozerov <vo...@gridgain.com> wrote:
>
> Resolvers were essential for DML because we had broken comparison semantics
> of binary objects. This is not the case now.
>
> Resolver as a whole is normal practice. E.g. it is implemented in .NET on
> core language level and widely used in many cases. Hazelcast has it as well
> AFAIK. So it is wrong to think that the whole idea is useless. Think of it
> as a comparator's brother.
>
> The only reason why we need to remove it is missing hash index in new
> architecture. It makes sense, as it is better to have AI 2.0 without them,
> than no AI 2.0 :-)
>
> 09 апр. 2017 г. 17:31 пользователь "Sergi Vladykin" <
> sergi.vladykin@gmail.com> написал:
>
>> I guess Resolvers were added to DML just because they already existed since
>> 1.9 and we were forced to support them in all the parts of our product.
>>
>> We have to stop this practice to add features without clear real life use
>> cases for them.
>>
>> Sergi
>>
>> 2017-04-09 17:00 GMT+03:00 Denis Magda <dm...@gridgain.com>:
>>
>>> Sergi, Vovan,
>>>
>>> Sorry for being annoying but I still didn't get an answer on whether the
>>> resolvers are the must for DML. The main reason why we made them up some
>>> time ago is to support specific DML use cases. However I can't recall the
>>> use cases.
>>>
>>> --
>>> Denis
>>>
>>> On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <sergi.vladykin@gmail.com
>>>
>>> wrote:
>>>
>>>> Ok, we need to do 2 things here:
>>>>
>>>> 1. Drop the resolvers from the source code.
>>>> 2. Write a good page in docs on "What makes a correct cache key".
>>>>
>>>> Who can do that?
>>>>
>>>> Sergi
>>>>
>>>> 2017-04-07 9:48 GMT+03:00 Sergi Vladykin <se...@gmail.com>:
>>>>
>>>>> It is possible to try adding support of comparison to Resolvers, but
>>> the
>>>>> whole approach looks wrong and for now it is better to get rid of it
>>>> while
>>>>> we have a chance to break compatibility.
>>>>>
>>>>> Sergi
>>>>>
>>>>> 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
>>>>> valentin.kulichenko@gmail.com>:
>>>>>
>>>>>> The discussion should've been started with that :) If supporting
>>>> resolvers
>>>>>> in new architecture is not possible or means too big effort, then
>> it's
>>>>>> definitely not worth it.
>>>>>>
>>>>>> -Val
>>>>>>
>>>>>> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <
>> vozerov@gridgain.com
>>>>
>>>>>> wrote:
>>>>>>
>>>>>>> Dima,
>>>>>>>
>>>>>>> Yes, they may explode some internals of our indexes.
>>>>>>>
>>>>>>> 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan" <
>>>>>>> dsetrakyan@apache.org> написал:
>>>>>>>
>>>>>>>> Guys,
>>>>>>>>
>>>>>>>> Isn't the main issue here that we cannot use the Identity
>>> Resolvers
>>>> in
>>>>>>>> BTrees in the 2.0 version? If yes, then we have to remove them
>> no
>>>>>> matter
>>>>>>>> what.
>>>>>>>>
>>>>>>>> D.
>>>>>>>>
>>>>>>>> On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
>>>>>> sergi.vladykin@gmail.com
>>>>>>>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Binary key representation is stable when we always have equal
>>>>>>> serialized
>>>>>>>>> bytes when the original keys are equal.
>>>>>>>>>
>>>>>>>>> Resolver allows you to have some extra info in the Key and
>> equal
>>>>>> Keys
>>>>>>>> will
>>>>>>>>> be serialized into different bytes, which is wrong.
>>>>>>>>>
>>>>>>>>> Look at the example what you can do with resolvers:
>>>>>>>>>
>>>>>>>>> We may have some data entry with fields a, b, c. Let's say the
>>>>>> unique
>>>>>>>> part
>>>>>>>>> here is `a` and it the only fields used in Key equals() and
>>>>>> hashCode().
>>>>>>>>> Still we may have the following layouts:
>>>>>>>>>
>>>>>>>>> 1. Ka -> Vbc
>>>>>>>>> 2. Kab -> Vc
>>>>>>>>> 3. Kabc -> Boolean.TRUE
>>>>>>>>>
>>>>>>>>> The only 1 is a correct layout, others are plain wrong
>> variants
>>>> (but
>>>>>>> they
>>>>>>>>> are still possible with Resolvers) because everything that
>> does
>>>> not
>>>>>>> make
>>>>>>>>> Key unique must be in Value.
>>>>>>>>>
>>>>>>>>> We want to clearly state that if you have something in Key,
>> that
>>>> is
>>>>>> not
>>>>>>>>> part of equals(), then the Key is invalid and that stuff must
>> be
>>>> in
>>>>>>>> Value.
>>>>>>>>> This allows us to rely on binary representation of a Key to be
>>>>>> stable
>>>>>>> and
>>>>>>>>> have some more optimizations and code simplifications with
>>> respect
>>>>>> to
>>>>>>>> these
>>>>>>>>> assumptions.
>>>>>>>>>
>>>>>>>>> Sergi
>>>>>>>>>
>>>>>>>>> 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
>>>>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>>>>
>>>>>>>>>> Even with my vast expirience I would never claim that I've
>>> seen
>>>>>>>>>> "everything" :)
>>>>>>>>>>
>>>>>>>>>> What do you mean by stable binary key representation and how
>>>>>>> resolvers
>>>>>>>>> make
>>>>>>>>>> it unstable?
>>>>>>>>>>
>>>>>>>>>> -Val
>>>>>>>>>>
>>>>>>>>>> On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Val,
>>>>>>>>>>>
>>>>>>>>>>> I know that you have really vast experience in Ignite
>>>>>> deployments
>>>>>>> and
>>>>>>>>>>> probably saw everything that can happen. Did you ever see
>>>>>> identity
>>>>>>>>>>> resolvers use in real life? I guess no.
>>>>>>>>>>>
>>>>>>>>>>> Hibernate example is bad here, because if their key is
>>>> unstable
>>>>>>>> across
>>>>>>>>>>> multiple JVMs, it means that it was not designed for
>>>> distributed
>>>>>>>>> caches a
>>>>>>>>>>> priori.
>>>>>>>>>>>
>>>>>>>>>>> Also knowing in advance about stable binary key
>>> representation
>>>>>>> allows
>>>>>>>>> us
>>>>>>>>>> to
>>>>>>>>>>> apply additional optimizations, like comparing keys
>> without
>>>>>>> detaching
>>>>>>>>>> them
>>>>>>>>>>> from offheap memory.
>>>>>>>>>>>
>>>>>>>>>>> We always will be able to add this stuff back if we see
>>> users
>>>>>>> really
>>>>>>>>> need
>>>>>>>>>>> it. Let's remove it for 2.0.
>>>>>>>>>>>
>>>>>>>>>>> Sergi
>>>>>>>>>>>
>>>>>>>>>>> 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
>>>>>>>>>>> valentin.kulichenko@gmail.com>:
>>>>>>>>>>>
>>>>>>>>>>>> Alex,
>>>>>>>>>>>>
>>>>>>>>>>>> To be honest, I don't understand the reasoning behind
>> the
>>>>>>> removal.
>>>>>>>> I
>>>>>>>>>>> think
>>>>>>>>>>>> resolvers provide good flexibility for different corner
>>>> cases
>>>>>> and
>>>>>>>>> it's
>>>>>>>>>> a
>>>>>>>>>>>> good thing to have them. Note that they can be applied
>> not
>>>>>> only
>>>>>>> to
>>>>>>>>>> cache
>>>>>>>>>>>> keys, but to any binary objects.
>>>>>>>>>>>>
>>>>>>>>>>>> Hibernate issue is actually a good example of such use
>>> case.
>>>>>> The
>>>>>>>> fact
>>>>>>>>>>> that
>>>>>>>>>>>> we found an alternative solution doesn't actually mean
>>>>>> anything,
>>>>>>>>>> because
>>>>>>>>>>>> what if this happened not in our module, but in user's
>>>>>>> application?
>>>>>>>>>>>> Unfortunately, we can't predict everything.
>>>>>>>>>>>>
>>>>>>>>>>>> Error proneness is not a very strong argument either,
>>>> because
>>>>>> in
>>>>>>> my
>>>>>>>>>> view
>>>>>>>>>>>> these resolvers are as much error prone as
>> BinaryIdMapper,
>>>> for
>>>>>>>>> example.
>>>>>>>>>>>>
>>>>>>>>>>>> -Val
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Denis,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Can you suggest a use-case where identity resolver is
>>>> needed
>>>>>>>> (given
>>>>>>>>>>> that
>>>>>>>>>>>> we
>>>>>>>>>>>>> agree that a key must contain only valuable fields)?
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2017-04-05 22:08 GMT+03:00 Denis Magda <
>>> dmagda@apache.org
>>>>> :
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Where do you want to remove the identity resolvers
>>> from?
>>>>>> If
>>>>>>>> it’s
>>>>>>>>>>>> related
>>>>>>>>>>>>>> to the internals of Hibernate module then it’s fine
>>> but
>>>> if
>>>>>>> you
>>>>>>>>>>> suggest
>>>>>>>>>>>>>> removing identity resolvers public interfaces then
>> it
>>>>>> might
>>>>>>> be
>>>>>>>> a
>>>>>>>>>>> haste
>>>>>>>>>>>>>> decision.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> —
>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
>>>>>>>>>>>>> alexey.goncharuk@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +1, I see no other reasons to keep it.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
>>>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> +1
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Lets drop them.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
>>>>>>>>>>>>>>>> dmitriy.govorukhin@gmail.com>
>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi guys, i implemented proxy for IgniteCache in
>>>>>> hibernate
>>>>>>>>>>>>> integration,
>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>> proxy transformate cacheKey to our key wrapper,
>>>> leaves
>>>>>>> only
>>>>>>>>>>>> required
>>>>>>>>>>>>>>>>> field. I think we can remove identity resolve,
>> it
>>>>>> should
>>>>>>>> not
>>>>>>>>>>> broke
>>>>>>>>>>>>>>>>> integration with hibernate. Any objections?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin
>>>> Kulichenko
>>>>>> <
>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I'm not saying there is no alternative
>> solution.
>>>> But
>>>>>>> let's
>>>>>>>>>>>> implement
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> prove that it works first, and remove resolvers
>>>> only
>>>>>>> after
>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> -Val
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi
>> Vladykin
>>> <
>>>>>>>>>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Guys, nothing is impossible if you know a bit
>>>> about
>>>>>>>>>> reflection
>>>>>>>>>>> in
>>>>>>>>>>>>>>>> Java
>>>>>>>>>>>>>>>>> :)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> We had a look at the CacheKey class and it is
>>>> easily
>>>>>>>>>>> replaceable.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
>>>>>>>>>>>>> dsetrakyan@apache.org
>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin
>>>>>> Kulichenko
>>>>>>> <
>>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> "Hibernate key" is the CacheKey class I was
>>>>>> referring
>>>>>>>> to.
>>>>>>>>>>> It's
>>>>>>>>>>>>>>>>>> provided
>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>> Hibernate, not by user and not by us. So I'm
>>> not
>>>>>> sure
>>>>>>>>> it's
>>>>>>>>>>>>>>>> possible
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> replace it.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> If it is impossible to replace or get rid of
>>> the
>>>>>>>> Hibernate
>>>>>>>>>>> key,
>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>> discussion valid at all?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
Re: Stable binary key representation
Posted by Vladimir Ozerov <vo...@gridgain.com>.
Resolvers were essential for DML because we had broken comparison semantics
of binary objects. This is not the case now.
Resolver as a whole is normal practice. E.g. it is implemented in .NET on
core language level and widely used in many cases. Hazelcast has it as well
AFAIK. So it is wrong to think that the whole idea is useless. Think of it
as a comparator's brother.
The only reason why we need to remove it is missing hash index in new
architecture. It makes sense, as it is better to have AI 2.0 without them,
than no AI 2.0 :-)
09 апр. 2017 г. 17:31 пользователь "Sergi Vladykin" <
sergi.vladykin@gmail.com> написал:
> I guess Resolvers were added to DML just because they already existed since
> 1.9 and we were forced to support them in all the parts of our product.
>
> We have to stop this practice to add features without clear real life use
> cases for them.
>
> Sergi
>
> 2017-04-09 17:00 GMT+03:00 Denis Magda <dm...@gridgain.com>:
>
> > Sergi, Vovan,
> >
> > Sorry for being annoying but I still didn't get an answer on whether the
> > resolvers are the must for DML. The main reason why we made them up some
> > time ago is to support specific DML use cases. However I can't recall the
> > use cases.
> >
> > --
> > Denis
> >
> > On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <sergi.vladykin@gmail.com
> >
> > wrote:
> >
> > > Ok, we need to do 2 things here:
> > >
> > > 1. Drop the resolvers from the source code.
> > > 2. Write a good page in docs on "What makes a correct cache key".
> > >
> > > Who can do that?
> > >
> > > Sergi
> > >
> > > 2017-04-07 9:48 GMT+03:00 Sergi Vladykin <se...@gmail.com>:
> > >
> > > > It is possible to try adding support of comparison to Resolvers, but
> > the
> > > > whole approach looks wrong and for now it is better to get rid of it
> > > while
> > > > we have a chance to break compatibility.
> > > >
> > > > Sergi
> > > >
> > > > 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
> > > > valentin.kulichenko@gmail.com>:
> > > >
> > > >> The discussion should've been started with that :) If supporting
> > > resolvers
> > > >> in new architecture is not possible or means too big effort, then
> it's
> > > >> definitely not worth it.
> > > >>
> > > >> -Val
> > > >>
> > > >> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <
> vozerov@gridgain.com
> > >
> > > >> wrote:
> > > >>
> > > >> > Dima,
> > > >> >
> > > >> > Yes, they may explode some internals of our indexes.
> > > >> >
> > > >> > 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan" <
> > > >> > dsetrakyan@apache.org> написал:
> > > >> >
> > > >> > > Guys,
> > > >> > >
> > > >> > > Isn't the main issue here that we cannot use the Identity
> > Resolvers
> > > in
> > > >> > > BTrees in the 2.0 version? If yes, then we have to remove them
> no
> > > >> matter
> > > >> > > what.
> > > >> > >
> > > >> > > D.
> > > >> > >
> > > >> > > On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
> > > >> sergi.vladykin@gmail.com
> > > >> > >
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Binary key representation is stable when we always have equal
> > > >> > serialized
> > > >> > > > bytes when the original keys are equal.
> > > >> > > >
> > > >> > > > Resolver allows you to have some extra info in the Key and
> equal
> > > >> Keys
> > > >> > > will
> > > >> > > > be serialized into different bytes, which is wrong.
> > > >> > > >
> > > >> > > > Look at the example what you can do with resolvers:
> > > >> > > >
> > > >> > > > We may have some data entry with fields a, b, c. Let's say the
> > > >> unique
> > > >> > > part
> > > >> > > > here is `a` and it the only fields used in Key equals() and
> > > >> hashCode().
> > > >> > > > Still we may have the following layouts:
> > > >> > > >
> > > >> > > > 1. Ka -> Vbc
> > > >> > > > 2. Kab -> Vc
> > > >> > > > 3. Kabc -> Boolean.TRUE
> > > >> > > >
> > > >> > > > The only 1 is a correct layout, others are plain wrong
> variants
> > > (but
> > > >> > they
> > > >> > > > are still possible with Resolvers) because everything that
> does
> > > not
> > > >> > make
> > > >> > > > Key unique must be in Value.
> > > >> > > >
> > > >> > > > We want to clearly state that if you have something in Key,
> that
> > > is
> > > >> not
> > > >> > > > part of equals(), then the Key is invalid and that stuff must
> be
> > > in
> > > >> > > Value.
> > > >> > > > This allows us to rely on binary representation of a Key to be
> > > >> stable
> > > >> > and
> > > >> > > > have some more optimizations and code simplifications with
> > respect
> > > >> to
> > > >> > > these
> > > >> > > > assumptions.
> > > >> > > >
> > > >> > > > Sergi
> > > >> > > >
> > > >> > > > 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
> > > >> > > > valentin.kulichenko@gmail.com>:
> > > >> > > >
> > > >> > > > > Even with my vast expirience I would never claim that I've
> > seen
> > > >> > > > > "everything" :)
> > > >> > > > >
> > > >> > > > > What do you mean by stable binary key representation and how
> > > >> > resolvers
> > > >> > > > make
> > > >> > > > > it unstable?
> > > >> > > > >
> > > >> > > > > -Val
> > > >> > > > >
> > > >> > > > > On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
> > > >> > > sergi.vladykin@gmail.com
> > > >> > > > >
> > > >> > > > > wrote:
> > > >> > > > >
> > > >> > > > > > Val,
> > > >> > > > > >
> > > >> > > > > > I know that you have really vast experience in Ignite
> > > >> deployments
> > > >> > and
> > > >> > > > > > probably saw everything that can happen. Did you ever see
> > > >> identity
> > > >> > > > > > resolvers use in real life? I guess no.
> > > >> > > > > >
> > > >> > > > > > Hibernate example is bad here, because if their key is
> > > unstable
> > > >> > > across
> > > >> > > > > > multiple JVMs, it means that it was not designed for
> > > distributed
> > > >> > > > caches a
> > > >> > > > > > priori.
> > > >> > > > > >
> > > >> > > > > > Also knowing in advance about stable binary key
> > representation
> > > >> > allows
> > > >> > > > us
> > > >> > > > > to
> > > >> > > > > > apply additional optimizations, like comparing keys
> without
> > > >> > detaching
> > > >> > > > > them
> > > >> > > > > > from offheap memory.
> > > >> > > > > >
> > > >> > > > > > We always will be able to add this stuff back if we see
> > users
> > > >> > really
> > > >> > > > need
> > > >> > > > > > it. Let's remove it for 2.0.
> > > >> > > > > >
> > > >> > > > > > Sergi
> > > >> > > > > >
> > > >> > > > > > 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
> > > >> > > > > > valentin.kulichenko@gmail.com>:
> > > >> > > > > >
> > > >> > > > > > > Alex,
> > > >> > > > > > >
> > > >> > > > > > > To be honest, I don't understand the reasoning behind
> the
> > > >> > removal.
> > > >> > > I
> > > >> > > > > > think
> > > >> > > > > > > resolvers provide good flexibility for different corner
> > > cases
> > > >> and
> > > >> > > > it's
> > > >> > > > > a
> > > >> > > > > > > good thing to have them. Note that they can be applied
> not
> > > >> only
> > > >> > to
> > > >> > > > > cache
> > > >> > > > > > > keys, but to any binary objects.
> > > >> > > > > > >
> > > >> > > > > > > Hibernate issue is actually a good example of such use
> > case.
> > > >> The
> > > >> > > fact
> > > >> > > > > > that
> > > >> > > > > > > we found an alternative solution doesn't actually mean
> > > >> anything,
> > > >> > > > > because
> > > >> > > > > > > what if this happened not in our module, but in user's
> > > >> > application?
> > > >> > > > > > > Unfortunately, we can't predict everything.
> > > >> > > > > > >
> > > >> > > > > > > Error proneness is not a very strong argument either,
> > > because
> > > >> in
> > > >> > my
> > > >> > > > > view
> > > >> > > > > > > these resolvers are as much error prone as
> BinaryIdMapper,
> > > for
> > > >> > > > example.
> > > >> > > > > > >
> > > >> > > > > > > -Val
> > > >> > > > > > >
> > > >> > > > > > > On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
> > > >> > > > > > > alexey.goncharuk@gmail.com> wrote:
> > > >> > > > > > >
> > > >> > > > > > > > Denis,
> > > >> > > > > > > >
> > > >> > > > > > > > Can you suggest a use-case where identity resolver is
> > > needed
> > > >> > > (given
> > > >> > > > > > that
> > > >> > > > > > > we
> > > >> > > > > > > > agree that a key must contain only valuable fields)?
> > > >> > > > > > > >
> > > >> > > > > > > > 2017-04-05 22:08 GMT+03:00 Denis Magda <
> > dmagda@apache.org
> > > >:
> > > >> > > > > > > >
> > > >> > > > > > > > > Where do you want to remove the identity resolvers
> > from?
> > > >> If
> > > >> > > it’s
> > > >> > > > > > > related
> > > >> > > > > > > > > to the internals of Hibernate module then it’s fine
> > but
> > > if
> > > >> > you
> > > >> > > > > > suggest
> > > >> > > > > > > > > removing identity resolvers public interfaces then
> it
> > > >> might
> > > >> > be
> > > >> > > a
> > > >> > > > > > haste
> > > >> > > > > > > > > decision.
> > > >> > > > > > > > >
> > > >> > > > > > > > > —
> > > >> > > > > > > > > Denis
> > > >> > > > > > > > >
> > > >> > > > > > > > > > On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
> > > >> > > > > > > > alexey.goncharuk@gmail.com>
> > > >> > > > > > > > > wrote:
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > +1, I see no other reasons to keep it.
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
> > > >> > > > > > sergi.vladykin@gmail.com
> > > >> > > > > > > >:
> > > >> > > > > > > > > >
> > > >> > > > > > > > > >> +1
> > > >> > > > > > > > > >>
> > > >> > > > > > > > > >> Lets drop them.
> > > >> > > > > > > > > >>
> > > >> > > > > > > > > >> Sergi
> > > >> > > > > > > > > >>
> > > >> > > > > > > > > >> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
> > > >> > > > > > > > > >> dmitriy.govorukhin@gmail.com>
> > > >> > > > > > > > > >> :
> > > >> > > > > > > > > >>
> > > >> > > > > > > > > >>> Hi guys, i implemented proxy for IgniteCache in
> > > >> hibernate
> > > >> > > > > > > > integration,
> > > >> > > > > > > > > >> this
> > > >> > > > > > > > > >>> proxy transformate cacheKey to our key wrapper,
> > > leaves
> > > >> > only
> > > >> > > > > > > required
> > > >> > > > > > > > > >>> field. I think we can remove identity resolve,
> it
> > > >> should
> > > >> > > not
> > > >> > > > > > broke
> > > >> > > > > > > > > >>> integration with hibernate. Any objections?
> > > >> > > > > > > > > >>>
> > > >> > > > > > > > > >>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin
> > > Kulichenko
> > > >> <
> > > >> > > > > > > > > >>> valentin.kulichenko@gmail.com> wrote:
> > > >> > > > > > > > > >>>
> > > >> > > > > > > > > >>>> I'm not saying there is no alternative
> solution.
> > > But
> > > >> > let's
> > > >> > > > > > > implement
> > > >> > > > > > > > > it
> > > >> > > > > > > > > >>> and
> > > >> > > > > > > > > >>>> prove that it works first, and remove resolvers
> > > only
> > > >> > after
> > > >> > > > > that.
> > > >> > > > > > > > > >>>>
> > > >> > > > > > > > > >>>> -Val
> > > >> > > > > > > > > >>>>
> > > >> > > > > > > > > >>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi
> Vladykin
> > <
> > > >> > > > > > > > > >>> sergi.vladykin@gmail.com
> > > >> > > > > > > > > >>>>>
> > > >> > > > > > > > > >>>> wrote:
> > > >> > > > > > > > > >>>>
> > > >> > > > > > > > > >>>>> Guys, nothing is impossible if you know a bit
> > > about
> > > >> > > > > reflection
> > > >> > > > > > in
> > > >> > > > > > > > > >> Java
> > > >> > > > > > > > > >>> :)
> > > >> > > > > > > > > >>>>>
> > > >> > > > > > > > > >>>>> We had a look at the CacheKey class and it is
> > > easily
> > > >> > > > > > replaceable.
> > > >> > > > > > > > > >>>>>
> > > >> > > > > > > > > >>>>> Sergi
> > > >> > > > > > > > > >>>>>
> > > >> > > > > > > > > >>>>>
> > > >> > > > > > > > > >>>>>
> > > >> > > > > > > > > >>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
> > > >> > > > > > > > dsetrakyan@apache.org
> > > >> > > > > > > > > >>> :
> > > >> > > > > > > > > >>>>>
> > > >> > > > > > > > > >>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin
> > > >> Kulichenko
> > > >> > <
> > > >> > > > > > > > > >>>>>> valentin.kulichenko@gmail.com> wrote:
> > > >> > > > > > > > > >>>>>>
> > > >> > > > > > > > > >>>>>>> "Hibernate key" is the CacheKey class I was
> > > >> referring
> > > >> > > to.
> > > >> > > > > > It's
> > > >> > > > > > > > > >>>> provided
> > > >> > > > > > > > > >>>>>> by
> > > >> > > > > > > > > >>>>>>> Hibernate, not by user and not by us. So I'm
> > not
> > > >> sure
> > > >> > > > it's
> > > >> > > > > > > > > >> possible
> > > >> > > > > > > > > >>>> to
> > > >> > > > > > > > > >>>>>>> replace it.
> > > >> > > > > > > > > >>>>>>>
> > > >> > > > > > > > > >>>>>>
> > > >> > > > > > > > > >>>>>> If it is impossible to replace or get rid of
> > the
> > > >> > > Hibernate
> > > >> > > > > > key,
> > > >> > > > > > > is
> > > >> > > > > > > > > >>> this
> > > >> > > > > > > > > >>>>>> discussion valid at all?
> > > >> > > > > > > > > >>>>>>
> > > >> > > > > > > > > >>>>>
> > > >> > > > > > > > > >>>>
> > > >> > > > > > > > > >>>
> > > >> > > > > > > > > >>
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>
Re: Stable binary key representation
Posted by Sergi Vladykin <se...@gmail.com>.
I guess Resolvers were added to DML just because they already existed since
1.9 and we were forced to support them in all the parts of our product.
We have to stop this practice to add features without clear real life use
cases for them.
Sergi
2017-04-09 17:00 GMT+03:00 Denis Magda <dm...@gridgain.com>:
> Sergi, Vovan,
>
> Sorry for being annoying but I still didn't get an answer on whether the
> resolvers are the must for DML. The main reason why we made them up some
> time ago is to support specific DML use cases. However I can't recall the
> use cases.
>
> --
> Denis
>
> On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <se...@gmail.com>
> wrote:
>
> > Ok, we need to do 2 things here:
> >
> > 1. Drop the resolvers from the source code.
> > 2. Write a good page in docs on "What makes a correct cache key".
> >
> > Who can do that?
> >
> > Sergi
> >
> > 2017-04-07 9:48 GMT+03:00 Sergi Vladykin <se...@gmail.com>:
> >
> > > It is possible to try adding support of comparison to Resolvers, but
> the
> > > whole approach looks wrong and for now it is better to get rid of it
> > while
> > > we have a chance to break compatibility.
> > >
> > > Sergi
> > >
> > > 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
> > > valentin.kulichenko@gmail.com>:
> > >
> > >> The discussion should've been started with that :) If supporting
> > resolvers
> > >> in new architecture is not possible or means too big effort, then it's
> > >> definitely not worth it.
> > >>
> > >> -Val
> > >>
> > >> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <vozerov@gridgain.com
> >
> > >> wrote:
> > >>
> > >> > Dima,
> > >> >
> > >> > Yes, they may explode some internals of our indexes.
> > >> >
> > >> > 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan" <
> > >> > dsetrakyan@apache.org> написал:
> > >> >
> > >> > > Guys,
> > >> > >
> > >> > > Isn't the main issue here that we cannot use the Identity
> Resolvers
> > in
> > >> > > BTrees in the 2.0 version? If yes, then we have to remove them no
> > >> matter
> > >> > > what.
> > >> > >
> > >> > > D.
> > >> > >
> > >> > > On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
> > >> sergi.vladykin@gmail.com
> > >> > >
> > >> > > wrote:
> > >> > >
> > >> > > > Binary key representation is stable when we always have equal
> > >> > serialized
> > >> > > > bytes when the original keys are equal.
> > >> > > >
> > >> > > > Resolver allows you to have some extra info in the Key and equal
> > >> Keys
> > >> > > will
> > >> > > > be serialized into different bytes, which is wrong.
> > >> > > >
> > >> > > > Look at the example what you can do with resolvers:
> > >> > > >
> > >> > > > We may have some data entry with fields a, b, c. Let's say the
> > >> unique
> > >> > > part
> > >> > > > here is `a` and it the only fields used in Key equals() and
> > >> hashCode().
> > >> > > > Still we may have the following layouts:
> > >> > > >
> > >> > > > 1. Ka -> Vbc
> > >> > > > 2. Kab -> Vc
> > >> > > > 3. Kabc -> Boolean.TRUE
> > >> > > >
> > >> > > > The only 1 is a correct layout, others are plain wrong variants
> > (but
> > >> > they
> > >> > > > are still possible with Resolvers) because everything that does
> > not
> > >> > make
> > >> > > > Key unique must be in Value.
> > >> > > >
> > >> > > > We want to clearly state that if you have something in Key, that
> > is
> > >> not
> > >> > > > part of equals(), then the Key is invalid and that stuff must be
> > in
> > >> > > Value.
> > >> > > > This allows us to rely on binary representation of a Key to be
> > >> stable
> > >> > and
> > >> > > > have some more optimizations and code simplifications with
> respect
> > >> to
> > >> > > these
> > >> > > > assumptions.
> > >> > > >
> > >> > > > Sergi
> > >> > > >
> > >> > > > 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
> > >> > > > valentin.kulichenko@gmail.com>:
> > >> > > >
> > >> > > > > Even with my vast expirience I would never claim that I've
> seen
> > >> > > > > "everything" :)
> > >> > > > >
> > >> > > > > What do you mean by stable binary key representation and how
> > >> > resolvers
> > >> > > > make
> > >> > > > > it unstable?
> > >> > > > >
> > >> > > > > -Val
> > >> > > > >
> > >> > > > > On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
> > >> > > sergi.vladykin@gmail.com
> > >> > > > >
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Val,
> > >> > > > > >
> > >> > > > > > I know that you have really vast experience in Ignite
> > >> deployments
> > >> > and
> > >> > > > > > probably saw everything that can happen. Did you ever see
> > >> identity
> > >> > > > > > resolvers use in real life? I guess no.
> > >> > > > > >
> > >> > > > > > Hibernate example is bad here, because if their key is
> > unstable
> > >> > > across
> > >> > > > > > multiple JVMs, it means that it was not designed for
> > distributed
> > >> > > > caches a
> > >> > > > > > priori.
> > >> > > > > >
> > >> > > > > > Also knowing in advance about stable binary key
> representation
> > >> > allows
> > >> > > > us
> > >> > > > > to
> > >> > > > > > apply additional optimizations, like comparing keys without
> > >> > detaching
> > >> > > > > them
> > >> > > > > > from offheap memory.
> > >> > > > > >
> > >> > > > > > We always will be able to add this stuff back if we see
> users
> > >> > really
> > >> > > > need
> > >> > > > > > it. Let's remove it for 2.0.
> > >> > > > > >
> > >> > > > > > Sergi
> > >> > > > > >
> > >> > > > > > 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
> > >> > > > > > valentin.kulichenko@gmail.com>:
> > >> > > > > >
> > >> > > > > > > Alex,
> > >> > > > > > >
> > >> > > > > > > To be honest, I don't understand the reasoning behind the
> > >> > removal.
> > >> > > I
> > >> > > > > > think
> > >> > > > > > > resolvers provide good flexibility for different corner
> > cases
> > >> and
> > >> > > > it's
> > >> > > > > a
> > >> > > > > > > good thing to have them. Note that they can be applied not
> > >> only
> > >> > to
> > >> > > > > cache
> > >> > > > > > > keys, but to any binary objects.
> > >> > > > > > >
> > >> > > > > > > Hibernate issue is actually a good example of such use
> case.
> > >> The
> > >> > > fact
> > >> > > > > > that
> > >> > > > > > > we found an alternative solution doesn't actually mean
> > >> anything,
> > >> > > > > because
> > >> > > > > > > what if this happened not in our module, but in user's
> > >> > application?
> > >> > > > > > > Unfortunately, we can't predict everything.
> > >> > > > > > >
> > >> > > > > > > Error proneness is not a very strong argument either,
> > because
> > >> in
> > >> > my
> > >> > > > > view
> > >> > > > > > > these resolvers are as much error prone as BinaryIdMapper,
> > for
> > >> > > > example.
> > >> > > > > > >
> > >> > > > > > > -Val
> > >> > > > > > >
> > >> > > > > > > On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
> > >> > > > > > > alexey.goncharuk@gmail.com> wrote:
> > >> > > > > > >
> > >> > > > > > > > Denis,
> > >> > > > > > > >
> > >> > > > > > > > Can you suggest a use-case where identity resolver is
> > needed
> > >> > > (given
> > >> > > > > > that
> > >> > > > > > > we
> > >> > > > > > > > agree that a key must contain only valuable fields)?
> > >> > > > > > > >
> > >> > > > > > > > 2017-04-05 22:08 GMT+03:00 Denis Magda <
> dmagda@apache.org
> > >:
> > >> > > > > > > >
> > >> > > > > > > > > Where do you want to remove the identity resolvers
> from?
> > >> If
> > >> > > it’s
> > >> > > > > > > related
> > >> > > > > > > > > to the internals of Hibernate module then it’s fine
> but
> > if
> > >> > you
> > >> > > > > > suggest
> > >> > > > > > > > > removing identity resolvers public interfaces then it
> > >> might
> > >> > be
> > >> > > a
> > >> > > > > > haste
> > >> > > > > > > > > decision.
> > >> > > > > > > > >
> > >> > > > > > > > > —
> > >> > > > > > > > > Denis
> > >> > > > > > > > >
> > >> > > > > > > > > > On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
> > >> > > > > > > > alexey.goncharuk@gmail.com>
> > >> > > > > > > > > wrote:
> > >> > > > > > > > > >
> > >> > > > > > > > > > +1, I see no other reasons to keep it.
> > >> > > > > > > > > >
> > >> > > > > > > > > > 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
> > >> > > > > > sergi.vladykin@gmail.com
> > >> > > > > > > >:
> > >> > > > > > > > > >
> > >> > > > > > > > > >> +1
> > >> > > > > > > > > >>
> > >> > > > > > > > > >> Lets drop them.
> > >> > > > > > > > > >>
> > >> > > > > > > > > >> Sergi
> > >> > > > > > > > > >>
> > >> > > > > > > > > >> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
> > >> > > > > > > > > >> dmitriy.govorukhin@gmail.com>
> > >> > > > > > > > > >> :
> > >> > > > > > > > > >>
> > >> > > > > > > > > >>> Hi guys, i implemented proxy for IgniteCache in
> > >> hibernate
> > >> > > > > > > > integration,
> > >> > > > > > > > > >> this
> > >> > > > > > > > > >>> proxy transformate cacheKey to our key wrapper,
> > leaves
> > >> > only
> > >> > > > > > > required
> > >> > > > > > > > > >>> field. I think we can remove identity resolve, it
> > >> should
> > >> > > not
> > >> > > > > > broke
> > >> > > > > > > > > >>> integration with hibernate. Any objections?
> > >> > > > > > > > > >>>
> > >> > > > > > > > > >>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin
> > Kulichenko
> > >> <
> > >> > > > > > > > > >>> valentin.kulichenko@gmail.com> wrote:
> > >> > > > > > > > > >>>
> > >> > > > > > > > > >>>> I'm not saying there is no alternative solution.
> > But
> > >> > let's
> > >> > > > > > > implement
> > >> > > > > > > > > it
> > >> > > > > > > > > >>> and
> > >> > > > > > > > > >>>> prove that it works first, and remove resolvers
> > only
> > >> > after
> > >> > > > > that.
> > >> > > > > > > > > >>>>
> > >> > > > > > > > > >>>> -Val
> > >> > > > > > > > > >>>>
> > >> > > > > > > > > >>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi Vladykin
> <
> > >> > > > > > > > > >>> sergi.vladykin@gmail.com
> > >> > > > > > > > > >>>>>
> > >> > > > > > > > > >>>> wrote:
> > >> > > > > > > > > >>>>
> > >> > > > > > > > > >>>>> Guys, nothing is impossible if you know a bit
> > about
> > >> > > > > reflection
> > >> > > > > > in
> > >> > > > > > > > > >> Java
> > >> > > > > > > > > >>> :)
> > >> > > > > > > > > >>>>>
> > >> > > > > > > > > >>>>> We had a look at the CacheKey class and it is
> > easily
> > >> > > > > > replaceable.
> > >> > > > > > > > > >>>>>
> > >> > > > > > > > > >>>>> Sergi
> > >> > > > > > > > > >>>>>
> > >> > > > > > > > > >>>>>
> > >> > > > > > > > > >>>>>
> > >> > > > > > > > > >>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
> > >> > > > > > > > dsetrakyan@apache.org
> > >> > > > > > > > > >>> :
> > >> > > > > > > > > >>>>>
> > >> > > > > > > > > >>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin
> > >> Kulichenko
> > >> > <
> > >> > > > > > > > > >>>>>> valentin.kulichenko@gmail.com> wrote:
> > >> > > > > > > > > >>>>>>
> > >> > > > > > > > > >>>>>>> "Hibernate key" is the CacheKey class I was
> > >> referring
> > >> > > to.
> > >> > > > > > It's
> > >> > > > > > > > > >>>> provided
> > >> > > > > > > > > >>>>>> by
> > >> > > > > > > > > >>>>>>> Hibernate, not by user and not by us. So I'm
> not
> > >> sure
> > >> > > > it's
> > >> > > > > > > > > >> possible
> > >> > > > > > > > > >>>> to
> > >> > > > > > > > > >>>>>>> replace it.
> > >> > > > > > > > > >>>>>>>
> > >> > > > > > > > > >>>>>>
> > >> > > > > > > > > >>>>>> If it is impossible to replace or get rid of
> the
> > >> > > Hibernate
> > >> > > > > > key,
> > >> > > > > > > is
> > >> > > > > > > > > >>> this
> > >> > > > > > > > > >>>>>> discussion valid at all?
> > >> > > > > > > > > >>>>>>
> > >> > > > > > > > > >>>>>
> > >> > > > > > > > > >>>>
> > >> > > > > > > > > >>>
> > >> > > > > > > > > >>
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>
Re: Stable binary key representation
Posted by Denis Magda <dm...@gridgain.com>.
Sergi, Vovan,
Sorry for being annoying but I still didn't get an answer on whether the
resolvers are the must for DML. The main reason why we made them up some
time ago is to support specific DML use cases. However I can't recall the
use cases.
--
Denis
On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <se...@gmail.com>
wrote:
> Ok, we need to do 2 things here:
>
> 1. Drop the resolvers from the source code.
> 2. Write a good page in docs on "What makes a correct cache key".
>
> Who can do that?
>
> Sergi
>
> 2017-04-07 9:48 GMT+03:00 Sergi Vladykin <se...@gmail.com>:
>
> > It is possible to try adding support of comparison to Resolvers, but the
> > whole approach looks wrong and for now it is better to get rid of it
> while
> > we have a chance to break compatibility.
> >
> > Sergi
> >
> > 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
> > valentin.kulichenko@gmail.com>:
> >
> >> The discussion should've been started with that :) If supporting
> resolvers
> >> in new architecture is not possible or means too big effort, then it's
> >> definitely not worth it.
> >>
> >> -Val
> >>
> >> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <vo...@gridgain.com>
> >> wrote:
> >>
> >> > Dima,
> >> >
> >> > Yes, they may explode some internals of our indexes.
> >> >
> >> > 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan" <
> >> > dsetrakyan@apache.org> написал:
> >> >
> >> > > Guys,
> >> > >
> >> > > Isn't the main issue here that we cannot use the Identity Resolvers
> in
> >> > > BTrees in the 2.0 version? If yes, then we have to remove them no
> >> matter
> >> > > what.
> >> > >
> >> > > D.
> >> > >
> >> > > On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
> >> sergi.vladykin@gmail.com
> >> > >
> >> > > wrote:
> >> > >
> >> > > > Binary key representation is stable when we always have equal
> >> > serialized
> >> > > > bytes when the original keys are equal.
> >> > > >
> >> > > > Resolver allows you to have some extra info in the Key and equal
> >> Keys
> >> > > will
> >> > > > be serialized into different bytes, which is wrong.
> >> > > >
> >> > > > Look at the example what you can do with resolvers:
> >> > > >
> >> > > > We may have some data entry with fields a, b, c. Let's say the
> >> unique
> >> > > part
> >> > > > here is `a` and it the only fields used in Key equals() and
> >> hashCode().
> >> > > > Still we may have the following layouts:
> >> > > >
> >> > > > 1. Ka -> Vbc
> >> > > > 2. Kab -> Vc
> >> > > > 3. Kabc -> Boolean.TRUE
> >> > > >
> >> > > > The only 1 is a correct layout, others are plain wrong variants
> (but
> >> > they
> >> > > > are still possible with Resolvers) because everything that does
> not
> >> > make
> >> > > > Key unique must be in Value.
> >> > > >
> >> > > > We want to clearly state that if you have something in Key, that
> is
> >> not
> >> > > > part of equals(), then the Key is invalid and that stuff must be
> in
> >> > > Value.
> >> > > > This allows us to rely on binary representation of a Key to be
> >> stable
> >> > and
> >> > > > have some more optimizations and code simplifications with respect
> >> to
> >> > > these
> >> > > > assumptions.
> >> > > >
> >> > > > Sergi
> >> > > >
> >> > > > 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
> >> > > > valentin.kulichenko@gmail.com>:
> >> > > >
> >> > > > > Even with my vast expirience I would never claim that I've seen
> >> > > > > "everything" :)
> >> > > > >
> >> > > > > What do you mean by stable binary key representation and how
> >> > resolvers
> >> > > > make
> >> > > > > it unstable?
> >> > > > >
> >> > > > > -Val
> >> > > > >
> >> > > > > On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
> >> > > sergi.vladykin@gmail.com
> >> > > > >
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Val,
> >> > > > > >
> >> > > > > > I know that you have really vast experience in Ignite
> >> deployments
> >> > and
> >> > > > > > probably saw everything that can happen. Did you ever see
> >> identity
> >> > > > > > resolvers use in real life? I guess no.
> >> > > > > >
> >> > > > > > Hibernate example is bad here, because if their key is
> unstable
> >> > > across
> >> > > > > > multiple JVMs, it means that it was not designed for
> distributed
> >> > > > caches a
> >> > > > > > priori.
> >> > > > > >
> >> > > > > > Also knowing in advance about stable binary key representation
> >> > allows
> >> > > > us
> >> > > > > to
> >> > > > > > apply additional optimizations, like comparing keys without
> >> > detaching
> >> > > > > them
> >> > > > > > from offheap memory.
> >> > > > > >
> >> > > > > > We always will be able to add this stuff back if we see users
> >> > really
> >> > > > need
> >> > > > > > it. Let's remove it for 2.0.
> >> > > > > >
> >> > > > > > Sergi
> >> > > > > >
> >> > > > > > 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
> >> > > > > > valentin.kulichenko@gmail.com>:
> >> > > > > >
> >> > > > > > > Alex,
> >> > > > > > >
> >> > > > > > > To be honest, I don't understand the reasoning behind the
> >> > removal.
> >> > > I
> >> > > > > > think
> >> > > > > > > resolvers provide good flexibility for different corner
> cases
> >> and
> >> > > > it's
> >> > > > > a
> >> > > > > > > good thing to have them. Note that they can be applied not
> >> only
> >> > to
> >> > > > > cache
> >> > > > > > > keys, but to any binary objects.
> >> > > > > > >
> >> > > > > > > Hibernate issue is actually a good example of such use case.
> >> The
> >> > > fact
> >> > > > > > that
> >> > > > > > > we found an alternative solution doesn't actually mean
> >> anything,
> >> > > > > because
> >> > > > > > > what if this happened not in our module, but in user's
> >> > application?
> >> > > > > > > Unfortunately, we can't predict everything.
> >> > > > > > >
> >> > > > > > > Error proneness is not a very strong argument either,
> because
> >> in
> >> > my
> >> > > > > view
> >> > > > > > > these resolvers are as much error prone as BinaryIdMapper,
> for
> >> > > > example.
> >> > > > > > >
> >> > > > > > > -Val
> >> > > > > > >
> >> > > > > > > On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
> >> > > > > > > alexey.goncharuk@gmail.com> wrote:
> >> > > > > > >
> >> > > > > > > > Denis,
> >> > > > > > > >
> >> > > > > > > > Can you suggest a use-case where identity resolver is
> needed
> >> > > (given
> >> > > > > > that
> >> > > > > > > we
> >> > > > > > > > agree that a key must contain only valuable fields)?
> >> > > > > > > >
> >> > > > > > > > 2017-04-05 22:08 GMT+03:00 Denis Magda <dmagda@apache.org
> >:
> >> > > > > > > >
> >> > > > > > > > > Where do you want to remove the identity resolvers from?
> >> If
> >> > > it’s
> >> > > > > > > related
> >> > > > > > > > > to the internals of Hibernate module then it’s fine but
> if
> >> > you
> >> > > > > > suggest
> >> > > > > > > > > removing identity resolvers public interfaces then it
> >> might
> >> > be
> >> > > a
> >> > > > > > haste
> >> > > > > > > > > decision.
> >> > > > > > > > >
> >> > > > > > > > > —
> >> > > > > > > > > Denis
> >> > > > > > > > >
> >> > > > > > > > > > On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
> >> > > > > > > > alexey.goncharuk@gmail.com>
> >> > > > > > > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > +1, I see no other reasons to keep it.
> >> > > > > > > > > >
> >> > > > > > > > > > 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
> >> > > > > > sergi.vladykin@gmail.com
> >> > > > > > > >:
> >> > > > > > > > > >
> >> > > > > > > > > >> +1
> >> > > > > > > > > >>
> >> > > > > > > > > >> Lets drop them.
> >> > > > > > > > > >>
> >> > > > > > > > > >> Sergi
> >> > > > > > > > > >>
> >> > > > > > > > > >> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
> >> > > > > > > > > >> dmitriy.govorukhin@gmail.com>
> >> > > > > > > > > >> :
> >> > > > > > > > > >>
> >> > > > > > > > > >>> Hi guys, i implemented proxy for IgniteCache in
> >> hibernate
> >> > > > > > > > integration,
> >> > > > > > > > > >> this
> >> > > > > > > > > >>> proxy transformate cacheKey to our key wrapper,
> leaves
> >> > only
> >> > > > > > > required
> >> > > > > > > > > >>> field. I think we can remove identity resolve, it
> >> should
> >> > > not
> >> > > > > > broke
> >> > > > > > > > > >>> integration with hibernate. Any objections?
> >> > > > > > > > > >>>
> >> > > > > > > > > >>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin
> Kulichenko
> >> <
> >> > > > > > > > > >>> valentin.kulichenko@gmail.com> wrote:
> >> > > > > > > > > >>>
> >> > > > > > > > > >>>> I'm not saying there is no alternative solution.
> But
> >> > let's
> >> > > > > > > implement
> >> > > > > > > > > it
> >> > > > > > > > > >>> and
> >> > > > > > > > > >>>> prove that it works first, and remove resolvers
> only
> >> > after
> >> > > > > that.
> >> > > > > > > > > >>>>
> >> > > > > > > > > >>>> -Val
> >> > > > > > > > > >>>>
> >> > > > > > > > > >>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi Vladykin <
> >> > > > > > > > > >>> sergi.vladykin@gmail.com
> >> > > > > > > > > >>>>>
> >> > > > > > > > > >>>> wrote:
> >> > > > > > > > > >>>>
> >> > > > > > > > > >>>>> Guys, nothing is impossible if you know a bit
> about
> >> > > > > reflection
> >> > > > > > in
> >> > > > > > > > > >> Java
> >> > > > > > > > > >>> :)
> >> > > > > > > > > >>>>>
> >> > > > > > > > > >>>>> We had a look at the CacheKey class and it is
> easily
> >> > > > > > replaceable.
> >> > > > > > > > > >>>>>
> >> > > > > > > > > >>>>> Sergi
> >> > > > > > > > > >>>>>
> >> > > > > > > > > >>>>>
> >> > > > > > > > > >>>>>
> >> > > > > > > > > >>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
> >> > > > > > > > dsetrakyan@apache.org
> >> > > > > > > > > >>> :
> >> > > > > > > > > >>>>>
> >> > > > > > > > > >>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin
> >> Kulichenko
> >> > <
> >> > > > > > > > > >>>>>> valentin.kulichenko@gmail.com> wrote:
> >> > > > > > > > > >>>>>>
> >> > > > > > > > > >>>>>>> "Hibernate key" is the CacheKey class I was
> >> referring
> >> > > to.
> >> > > > > > It's
> >> > > > > > > > > >>>> provided
> >> > > > > > > > > >>>>>> by
> >> > > > > > > > > >>>>>>> Hibernate, not by user and not by us. So I'm not
> >> sure
> >> > > > it's
> >> > > > > > > > > >> possible
> >> > > > > > > > > >>>> to
> >> > > > > > > > > >>>>>>> replace it.
> >> > > > > > > > > >>>>>>>
> >> > > > > > > > > >>>>>>
> >> > > > > > > > > >>>>>> If it is impossible to replace or get rid of the
> >> > > Hibernate
> >> > > > > > key,
> >> > > > > > > is
> >> > > > > > > > > >>> this
> >> > > > > > > > > >>>>>> discussion valid at all?
> >> > > > > > > > > >>>>>>
> >> > > > > > > > > >>>>>
> >> > > > > > > > > >>>>
> >> > > > > > > > > >>>
> >> > > > > > > > > >>
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
Re: Stable binary key representation
Posted by Sergi Vladykin <se...@gmail.com>.
Ok, we need to do 2 things here:
1. Drop the resolvers from the source code.
2. Write a good page in docs on "What makes a correct cache key".
Who can do that?
Sergi
2017-04-07 9:48 GMT+03:00 Sergi Vladykin <se...@gmail.com>:
> It is possible to try adding support of comparison to Resolvers, but the
> whole approach looks wrong and for now it is better to get rid of it while
> we have a chance to break compatibility.
>
> Sergi
>
> 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
> valentin.kulichenko@gmail.com>:
>
>> The discussion should've been started with that :) If supporting resolvers
>> in new architecture is not possible or means too big effort, then it's
>> definitely not worth it.
>>
>> -Val
>>
>> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <vo...@gridgain.com>
>> wrote:
>>
>> > Dima,
>> >
>> > Yes, they may explode some internals of our indexes.
>> >
>> > 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan" <
>> > dsetrakyan@apache.org> написал:
>> >
>> > > Guys,
>> > >
>> > > Isn't the main issue here that we cannot use the Identity Resolvers in
>> > > BTrees in the 2.0 version? If yes, then we have to remove them no
>> matter
>> > > what.
>> > >
>> > > D.
>> > >
>> > > On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
>> sergi.vladykin@gmail.com
>> > >
>> > > wrote:
>> > >
>> > > > Binary key representation is stable when we always have equal
>> > serialized
>> > > > bytes when the original keys are equal.
>> > > >
>> > > > Resolver allows you to have some extra info in the Key and equal
>> Keys
>> > > will
>> > > > be serialized into different bytes, which is wrong.
>> > > >
>> > > > Look at the example what you can do with resolvers:
>> > > >
>> > > > We may have some data entry with fields a, b, c. Let's say the
>> unique
>> > > part
>> > > > here is `a` and it the only fields used in Key equals() and
>> hashCode().
>> > > > Still we may have the following layouts:
>> > > >
>> > > > 1. Ka -> Vbc
>> > > > 2. Kab -> Vc
>> > > > 3. Kabc -> Boolean.TRUE
>> > > >
>> > > > The only 1 is a correct layout, others are plain wrong variants (but
>> > they
>> > > > are still possible with Resolvers) because everything that does not
>> > make
>> > > > Key unique must be in Value.
>> > > >
>> > > > We want to clearly state that if you have something in Key, that is
>> not
>> > > > part of equals(), then the Key is invalid and that stuff must be in
>> > > Value.
>> > > > This allows us to rely on binary representation of a Key to be
>> stable
>> > and
>> > > > have some more optimizations and code simplifications with respect
>> to
>> > > these
>> > > > assumptions.
>> > > >
>> > > > Sergi
>> > > >
>> > > > 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
>> > > > valentin.kulichenko@gmail.com>:
>> > > >
>> > > > > Even with my vast expirience I would never claim that I've seen
>> > > > > "everything" :)
>> > > > >
>> > > > > What do you mean by stable binary key representation and how
>> > resolvers
>> > > > make
>> > > > > it unstable?
>> > > > >
>> > > > > -Val
>> > > > >
>> > > > > On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
>> > > sergi.vladykin@gmail.com
>> > > > >
>> > > > > wrote:
>> > > > >
>> > > > > > Val,
>> > > > > >
>> > > > > > I know that you have really vast experience in Ignite
>> deployments
>> > and
>> > > > > > probably saw everything that can happen. Did you ever see
>> identity
>> > > > > > resolvers use in real life? I guess no.
>> > > > > >
>> > > > > > Hibernate example is bad here, because if their key is unstable
>> > > across
>> > > > > > multiple JVMs, it means that it was not designed for distributed
>> > > > caches a
>> > > > > > priori.
>> > > > > >
>> > > > > > Also knowing in advance about stable binary key representation
>> > allows
>> > > > us
>> > > > > to
>> > > > > > apply additional optimizations, like comparing keys without
>> > detaching
>> > > > > them
>> > > > > > from offheap memory.
>> > > > > >
>> > > > > > We always will be able to add this stuff back if we see users
>> > really
>> > > > need
>> > > > > > it. Let's remove it for 2.0.
>> > > > > >
>> > > > > > Sergi
>> > > > > >
>> > > > > > 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
>> > > > > > valentin.kulichenko@gmail.com>:
>> > > > > >
>> > > > > > > Alex,
>> > > > > > >
>> > > > > > > To be honest, I don't understand the reasoning behind the
>> > removal.
>> > > I
>> > > > > > think
>> > > > > > > resolvers provide good flexibility for different corner cases
>> and
>> > > > it's
>> > > > > a
>> > > > > > > good thing to have them. Note that they can be applied not
>> only
>> > to
>> > > > > cache
>> > > > > > > keys, but to any binary objects.
>> > > > > > >
>> > > > > > > Hibernate issue is actually a good example of such use case.
>> The
>> > > fact
>> > > > > > that
>> > > > > > > we found an alternative solution doesn't actually mean
>> anything,
>> > > > > because
>> > > > > > > what if this happened not in our module, but in user's
>> > application?
>> > > > > > > Unfortunately, we can't predict everything.
>> > > > > > >
>> > > > > > > Error proneness is not a very strong argument either, because
>> in
>> > my
>> > > > > view
>> > > > > > > these resolvers are as much error prone as BinaryIdMapper, for
>> > > > example.
>> > > > > > >
>> > > > > > > -Val
>> > > > > > >
>> > > > > > > On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
>> > > > > > > alexey.goncharuk@gmail.com> wrote:
>> > > > > > >
>> > > > > > > > Denis,
>> > > > > > > >
>> > > > > > > > Can you suggest a use-case where identity resolver is needed
>> > > (given
>> > > > > > that
>> > > > > > > we
>> > > > > > > > agree that a key must contain only valuable fields)?
>> > > > > > > >
>> > > > > > > > 2017-04-05 22:08 GMT+03:00 Denis Magda <dm...@apache.org>:
>> > > > > > > >
>> > > > > > > > > Where do you want to remove the identity resolvers from?
>> If
>> > > it’s
>> > > > > > > related
>> > > > > > > > > to the internals of Hibernate module then it’s fine but if
>> > you
>> > > > > > suggest
>> > > > > > > > > removing identity resolvers public interfaces then it
>> might
>> > be
>> > > a
>> > > > > > haste
>> > > > > > > > > decision.
>> > > > > > > > >
>> > > > > > > > > —
>> > > > > > > > > Denis
>> > > > > > > > >
>> > > > > > > > > > On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
>> > > > > > > > alexey.goncharuk@gmail.com>
>> > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > +1, I see no other reasons to keep it.
>> > > > > > > > > >
>> > > > > > > > > > 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
>> > > > > > sergi.vladykin@gmail.com
>> > > > > > > >:
>> > > > > > > > > >
>> > > > > > > > > >> +1
>> > > > > > > > > >>
>> > > > > > > > > >> Lets drop them.
>> > > > > > > > > >>
>> > > > > > > > > >> Sergi
>> > > > > > > > > >>
>> > > > > > > > > >> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
>> > > > > > > > > >> dmitriy.govorukhin@gmail.com>
>> > > > > > > > > >> :
>> > > > > > > > > >>
>> > > > > > > > > >>> Hi guys, i implemented proxy for IgniteCache in
>> hibernate
>> > > > > > > > integration,
>> > > > > > > > > >> this
>> > > > > > > > > >>> proxy transformate cacheKey to our key wrapper, leaves
>> > only
>> > > > > > > required
>> > > > > > > > > >>> field. I think we can remove identity resolve, it
>> should
>> > > not
>> > > > > > broke
>> > > > > > > > > >>> integration with hibernate. Any objections?
>> > > > > > > > > >>>
>> > > > > > > > > >>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin Kulichenko
>> <
>> > > > > > > > > >>> valentin.kulichenko@gmail.com> wrote:
>> > > > > > > > > >>>
>> > > > > > > > > >>>> I'm not saying there is no alternative solution. But
>> > let's
>> > > > > > > implement
>> > > > > > > > > it
>> > > > > > > > > >>> and
>> > > > > > > > > >>>> prove that it works first, and remove resolvers only
>> > after
>> > > > > that.
>> > > > > > > > > >>>>
>> > > > > > > > > >>>> -Val
>> > > > > > > > > >>>>
>> > > > > > > > > >>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi Vladykin <
>> > > > > > > > > >>> sergi.vladykin@gmail.com
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>> wrote:
>> > > > > > > > > >>>>
>> > > > > > > > > >>>>> Guys, nothing is impossible if you know a bit about
>> > > > > reflection
>> > > > > > in
>> > > > > > > > > >> Java
>> > > > > > > > > >>> :)
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>> We had a look at the CacheKey class and it is easily
>> > > > > > replaceable.
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>> Sergi
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
>> > > > > > > > dsetrakyan@apache.org
>> > > > > > > > > >>> :
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin
>> Kulichenko
>> > <
>> > > > > > > > > >>>>>> valentin.kulichenko@gmail.com> wrote:
>> > > > > > > > > >>>>>>
>> > > > > > > > > >>>>>>> "Hibernate key" is the CacheKey class I was
>> referring
>> > > to.
>> > > > > > It's
>> > > > > > > > > >>>> provided
>> > > > > > > > > >>>>>> by
>> > > > > > > > > >>>>>>> Hibernate, not by user and not by us. So I'm not
>> sure
>> > > > it's
>> > > > > > > > > >> possible
>> > > > > > > > > >>>> to
>> > > > > > > > > >>>>>>> replace it.
>> > > > > > > > > >>>>>>>
>> > > > > > > > > >>>>>>
>> > > > > > > > > >>>>>> If it is impossible to replace or get rid of the
>> > > Hibernate
>> > > > > > key,
>> > > > > > > is
>> > > > > > > > > >>> this
>> > > > > > > > > >>>>>> discussion valid at all?
>> > > > > > > > > >>>>>>
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>
>> > > > > > > > > >>>
>> > > > > > > > > >>
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
Re: Stable binary key representation
Posted by Sergi Vladykin <se...@gmail.com>.
It is possible to try adding support of comparison to Resolvers, but the
whole approach looks wrong and for now it is better to get rid of it while
we have a chance to break compatibility.
Sergi
2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <valentin.kulichenko@gmail.com
>:
> The discussion should've been started with that :) If supporting resolvers
> in new architecture is not possible or means too big effort, then it's
> definitely not worth it.
>
> -Val
>
> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <vo...@gridgain.com>
> wrote:
>
> > Dima,
> >
> > Yes, they may explode some internals of our indexes.
> >
> > 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan" <
> > dsetrakyan@apache.org> написал:
> >
> > > Guys,
> > >
> > > Isn't the main issue here that we cannot use the Identity Resolvers in
> > > BTrees in the 2.0 version? If yes, then we have to remove them no
> matter
> > > what.
> > >
> > > D.
> > >
> > > On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
> sergi.vladykin@gmail.com
> > >
> > > wrote:
> > >
> > > > Binary key representation is stable when we always have equal
> > serialized
> > > > bytes when the original keys are equal.
> > > >
> > > > Resolver allows you to have some extra info in the Key and equal Keys
> > > will
> > > > be serialized into different bytes, which is wrong.
> > > >
> > > > Look at the example what you can do with resolvers:
> > > >
> > > > We may have some data entry with fields a, b, c. Let's say the unique
> > > part
> > > > here is `a` and it the only fields used in Key equals() and
> hashCode().
> > > > Still we may have the following layouts:
> > > >
> > > > 1. Ka -> Vbc
> > > > 2. Kab -> Vc
> > > > 3. Kabc -> Boolean.TRUE
> > > >
> > > > The only 1 is a correct layout, others are plain wrong variants (but
> > they
> > > > are still possible with Resolvers) because everything that does not
> > make
> > > > Key unique must be in Value.
> > > >
> > > > We want to clearly state that if you have something in Key, that is
> not
> > > > part of equals(), then the Key is invalid and that stuff must be in
> > > Value.
> > > > This allows us to rely on binary representation of a Key to be stable
> > and
> > > > have some more optimizations and code simplifications with respect to
> > > these
> > > > assumptions.
> > > >
> > > > Sergi
> > > >
> > > > 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
> > > > valentin.kulichenko@gmail.com>:
> > > >
> > > > > Even with my vast expirience I would never claim that I've seen
> > > > > "everything" :)
> > > > >
> > > > > What do you mean by stable binary key representation and how
> > resolvers
> > > > make
> > > > > it unstable?
> > > > >
> > > > > -Val
> > > > >
> > > > > On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
> > > sergi.vladykin@gmail.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > Val,
> > > > > >
> > > > > > I know that you have really vast experience in Ignite deployments
> > and
> > > > > > probably saw everything that can happen. Did you ever see
> identity
> > > > > > resolvers use in real life? I guess no.
> > > > > >
> > > > > > Hibernate example is bad here, because if their key is unstable
> > > across
> > > > > > multiple JVMs, it means that it was not designed for distributed
> > > > caches a
> > > > > > priori.
> > > > > >
> > > > > > Also knowing in advance about stable binary key representation
> > allows
> > > > us
> > > > > to
> > > > > > apply additional optimizations, like comparing keys without
> > detaching
> > > > > them
> > > > > > from offheap memory.
> > > > > >
> > > > > > We always will be able to add this stuff back if we see users
> > really
> > > > need
> > > > > > it. Let's remove it for 2.0.
> > > > > >
> > > > > > Sergi
> > > > > >
> > > > > > 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
> > > > > > valentin.kulichenko@gmail.com>:
> > > > > >
> > > > > > > Alex,
> > > > > > >
> > > > > > > To be honest, I don't understand the reasoning behind the
> > removal.
> > > I
> > > > > > think
> > > > > > > resolvers provide good flexibility for different corner cases
> and
> > > > it's
> > > > > a
> > > > > > > good thing to have them. Note that they can be applied not only
> > to
> > > > > cache
> > > > > > > keys, but to any binary objects.
> > > > > > >
> > > > > > > Hibernate issue is actually a good example of such use case.
> The
> > > fact
> > > > > > that
> > > > > > > we found an alternative solution doesn't actually mean
> anything,
> > > > > because
> > > > > > > what if this happened not in our module, but in user's
> > application?
> > > > > > > Unfortunately, we can't predict everything.
> > > > > > >
> > > > > > > Error proneness is not a very strong argument either, because
> in
> > my
> > > > > view
> > > > > > > these resolvers are as much error prone as BinaryIdMapper, for
> > > > example.
> > > > > > >
> > > > > > > -Val
> > > > > > >
> > > > > > > On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
> > > > > > > alexey.goncharuk@gmail.com> wrote:
> > > > > > >
> > > > > > > > Denis,
> > > > > > > >
> > > > > > > > Can you suggest a use-case where identity resolver is needed
> > > (given
> > > > > > that
> > > > > > > we
> > > > > > > > agree that a key must contain only valuable fields)?
> > > > > > > >
> > > > > > > > 2017-04-05 22:08 GMT+03:00 Denis Magda <dm...@apache.org>:
> > > > > > > >
> > > > > > > > > Where do you want to remove the identity resolvers from? If
> > > it’s
> > > > > > > related
> > > > > > > > > to the internals of Hibernate module then it’s fine but if
> > you
> > > > > > suggest
> > > > > > > > > removing identity resolvers public interfaces then it might
> > be
> > > a
> > > > > > haste
> > > > > > > > > decision.
> > > > > > > > >
> > > > > > > > > —
> > > > > > > > > Denis
> > > > > > > > >
> > > > > > > > > > On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
> > > > > > > > alexey.goncharuk@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > +1, I see no other reasons to keep it.
> > > > > > > > > >
> > > > > > > > > > 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
> > > > > > sergi.vladykin@gmail.com
> > > > > > > >:
> > > > > > > > > >
> > > > > > > > > >> +1
> > > > > > > > > >>
> > > > > > > > > >> Lets drop them.
> > > > > > > > > >>
> > > > > > > > > >> Sergi
> > > > > > > > > >>
> > > > > > > > > >> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
> > > > > > > > > >> dmitriy.govorukhin@gmail.com>
> > > > > > > > > >> :
> > > > > > > > > >>
> > > > > > > > > >>> Hi guys, i implemented proxy for IgniteCache in
> hibernate
> > > > > > > > integration,
> > > > > > > > > >> this
> > > > > > > > > >>> proxy transformate cacheKey to our key wrapper, leaves
> > only
> > > > > > > required
> > > > > > > > > >>> field. I think we can remove identity resolve, it
> should
> > > not
> > > > > > broke
> > > > > > > > > >>> integration with hibernate. Any objections?
> > > > > > > > > >>>
> > > > > > > > > >>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin Kulichenko <
> > > > > > > > > >>> valentin.kulichenko@gmail.com> wrote:
> > > > > > > > > >>>
> > > > > > > > > >>>> I'm not saying there is no alternative solution. But
> > let's
> > > > > > > implement
> > > > > > > > > it
> > > > > > > > > >>> and
> > > > > > > > > >>>> prove that it works first, and remove resolvers only
> > after
> > > > > that.
> > > > > > > > > >>>>
> > > > > > > > > >>>> -Val
> > > > > > > > > >>>>
> > > > > > > > > >>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi Vladykin <
> > > > > > > > > >>> sergi.vladykin@gmail.com
> > > > > > > > > >>>>>
> > > > > > > > > >>>> wrote:
> > > > > > > > > >>>>
> > > > > > > > > >>>>> Guys, nothing is impossible if you know a bit about
> > > > > reflection
> > > > > > in
> > > > > > > > > >> Java
> > > > > > > > > >>> :)
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> We had a look at the CacheKey class and it is easily
> > > > > > replaceable.
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> Sergi
> > > > > > > > > >>>>>
> > > > > > > > > >>>>>
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
> > > > > > > > dsetrakyan@apache.org
> > > > > > > > > >>> :
> > > > > > > > > >>>>>
> > > > > > > > > >>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin
> Kulichenko
> > <
> > > > > > > > > >>>>>> valentin.kulichenko@gmail.com> wrote:
> > > > > > > > > >>>>>>
> > > > > > > > > >>>>>>> "Hibernate key" is the CacheKey class I was
> referring
> > > to.
> > > > > > It's
> > > > > > > > > >>>> provided
> > > > > > > > > >>>>>> by
> > > > > > > > > >>>>>>> Hibernate, not by user and not by us. So I'm not
> sure
> > > > it's
> > > > > > > > > >> possible
> > > > > > > > > >>>> to
> > > > > > > > > >>>>>>> replace it.
> > > > > > > > > >>>>>>>
> > > > > > > > > >>>>>>
> > > > > > > > > >>>>>> If it is impossible to replace or get rid of the
> > > Hibernate
> > > > > > key,
> > > > > > > is
> > > > > > > > > >>> this
> > > > > > > > > >>>>>> discussion valid at all?
> > > > > > > > > >>>>>>
> > > > > > > > > >>>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>
> > > > > > > > > >>
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Stable binary key representation
Posted by Valentin Kulichenko <va...@gmail.com>.
The discussion should've been started with that :) If supporting resolvers
in new architecture is not possible or means too big effort, then it's
definitely not worth it.
-Val
On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <vo...@gridgain.com>
wrote:
> Dima,
>
> Yes, they may explode some internals of our indexes.
>
> 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan" <
> dsetrakyan@apache.org> написал:
>
> > Guys,
> >
> > Isn't the main issue here that we cannot use the Identity Resolvers in
> > BTrees in the 2.0 version? If yes, then we have to remove them no matter
> > what.
> >
> > D.
> >
> > On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <sergi.vladykin@gmail.com
> >
> > wrote:
> >
> > > Binary key representation is stable when we always have equal
> serialized
> > > bytes when the original keys are equal.
> > >
> > > Resolver allows you to have some extra info in the Key and equal Keys
> > will
> > > be serialized into different bytes, which is wrong.
> > >
> > > Look at the example what you can do with resolvers:
> > >
> > > We may have some data entry with fields a, b, c. Let's say the unique
> > part
> > > here is `a` and it the only fields used in Key equals() and hashCode().
> > > Still we may have the following layouts:
> > >
> > > 1. Ka -> Vbc
> > > 2. Kab -> Vc
> > > 3. Kabc -> Boolean.TRUE
> > >
> > > The only 1 is a correct layout, others are plain wrong variants (but
> they
> > > are still possible with Resolvers) because everything that does not
> make
> > > Key unique must be in Value.
> > >
> > > We want to clearly state that if you have something in Key, that is not
> > > part of equals(), then the Key is invalid and that stuff must be in
> > Value.
> > > This allows us to rely on binary representation of a Key to be stable
> and
> > > have some more optimizations and code simplifications with respect to
> > these
> > > assumptions.
> > >
> > > Sergi
> > >
> > > 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
> > > valentin.kulichenko@gmail.com>:
> > >
> > > > Even with my vast expirience I would never claim that I've seen
> > > > "everything" :)
> > > >
> > > > What do you mean by stable binary key representation and how
> resolvers
> > > make
> > > > it unstable?
> > > >
> > > > -Val
> > > >
> > > > On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
> > sergi.vladykin@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > Val,
> > > > >
> > > > > I know that you have really vast experience in Ignite deployments
> and
> > > > > probably saw everything that can happen. Did you ever see identity
> > > > > resolvers use in real life? I guess no.
> > > > >
> > > > > Hibernate example is bad here, because if their key is unstable
> > across
> > > > > multiple JVMs, it means that it was not designed for distributed
> > > caches a
> > > > > priori.
> > > > >
> > > > > Also knowing in advance about stable binary key representation
> allows
> > > us
> > > > to
> > > > > apply additional optimizations, like comparing keys without
> detaching
> > > > them
> > > > > from offheap memory.
> > > > >
> > > > > We always will be able to add this stuff back if we see users
> really
> > > need
> > > > > it. Let's remove it for 2.0.
> > > > >
> > > > > Sergi
> > > > >
> > > > > 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
> > > > > valentin.kulichenko@gmail.com>:
> > > > >
> > > > > > Alex,
> > > > > >
> > > > > > To be honest, I don't understand the reasoning behind the
> removal.
> > I
> > > > > think
> > > > > > resolvers provide good flexibility for different corner cases and
> > > it's
> > > > a
> > > > > > good thing to have them. Note that they can be applied not only
> to
> > > > cache
> > > > > > keys, but to any binary objects.
> > > > > >
> > > > > > Hibernate issue is actually a good example of such use case. The
> > fact
> > > > > that
> > > > > > we found an alternative solution doesn't actually mean anything,
> > > > because
> > > > > > what if this happened not in our module, but in user's
> application?
> > > > > > Unfortunately, we can't predict everything.
> > > > > >
> > > > > > Error proneness is not a very strong argument either, because in
> my
> > > > view
> > > > > > these resolvers are as much error prone as BinaryIdMapper, for
> > > example.
> > > > > >
> > > > > > -Val
> > > > > >
> > > > > > On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
> > > > > > alexey.goncharuk@gmail.com> wrote:
> > > > > >
> > > > > > > Denis,
> > > > > > >
> > > > > > > Can you suggest a use-case where identity resolver is needed
> > (given
> > > > > that
> > > > > > we
> > > > > > > agree that a key must contain only valuable fields)?
> > > > > > >
> > > > > > > 2017-04-05 22:08 GMT+03:00 Denis Magda <dm...@apache.org>:
> > > > > > >
> > > > > > > > Where do you want to remove the identity resolvers from? If
> > it’s
> > > > > > related
> > > > > > > > to the internals of Hibernate module then it’s fine but if
> you
> > > > > suggest
> > > > > > > > removing identity resolvers public interfaces then it might
> be
> > a
> > > > > haste
> > > > > > > > decision.
> > > > > > > >
> > > > > > > > —
> > > > > > > > Denis
> > > > > > > >
> > > > > > > > > On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
> > > > > > > alexey.goncharuk@gmail.com>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > +1, I see no other reasons to keep it.
> > > > > > > > >
> > > > > > > > > 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
> > > > > sergi.vladykin@gmail.com
> > > > > > >:
> > > > > > > > >
> > > > > > > > >> +1
> > > > > > > > >>
> > > > > > > > >> Lets drop them.
> > > > > > > > >>
> > > > > > > > >> Sergi
> > > > > > > > >>
> > > > > > > > >> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
> > > > > > > > >> dmitriy.govorukhin@gmail.com>
> > > > > > > > >> :
> > > > > > > > >>
> > > > > > > > >>> Hi guys, i implemented proxy for IgniteCache in hibernate
> > > > > > > integration,
> > > > > > > > >> this
> > > > > > > > >>> proxy transformate cacheKey to our key wrapper, leaves
> only
> > > > > > required
> > > > > > > > >>> field. I think we can remove identity resolve, it should
> > not
> > > > > broke
> > > > > > > > >>> integration with hibernate. Any objections?
> > > > > > > > >>>
> > > > > > > > >>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin Kulichenko <
> > > > > > > > >>> valentin.kulichenko@gmail.com> wrote:
> > > > > > > > >>>
> > > > > > > > >>>> I'm not saying there is no alternative solution. But
> let's
> > > > > > implement
> > > > > > > > it
> > > > > > > > >>> and
> > > > > > > > >>>> prove that it works first, and remove resolvers only
> after
> > > > that.
> > > > > > > > >>>>
> > > > > > > > >>>> -Val
> > > > > > > > >>>>
> > > > > > > > >>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi Vladykin <
> > > > > > > > >>> sergi.vladykin@gmail.com
> > > > > > > > >>>>>
> > > > > > > > >>>> wrote:
> > > > > > > > >>>>
> > > > > > > > >>>>> Guys, nothing is impossible if you know a bit about
> > > > reflection
> > > > > in
> > > > > > > > >> Java
> > > > > > > > >>> :)
> > > > > > > > >>>>>
> > > > > > > > >>>>> We had a look at the CacheKey class and it is easily
> > > > > replaceable.
> > > > > > > > >>>>>
> > > > > > > > >>>>> Sergi
> > > > > > > > >>>>>
> > > > > > > > >>>>>
> > > > > > > > >>>>>
> > > > > > > > >>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
> > > > > > > dsetrakyan@apache.org
> > > > > > > > >>> :
> > > > > > > > >>>>>
> > > > > > > > >>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin Kulichenko
> <
> > > > > > > > >>>>>> valentin.kulichenko@gmail.com> wrote:
> > > > > > > > >>>>>>
> > > > > > > > >>>>>>> "Hibernate key" is the CacheKey class I was referring
> > to.
> > > > > It's
> > > > > > > > >>>> provided
> > > > > > > > >>>>>> by
> > > > > > > > >>>>>>> Hibernate, not by user and not by us. So I'm not sure
> > > it's
> > > > > > > > >> possible
> > > > > > > > >>>> to
> > > > > > > > >>>>>>> replace it.
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> If it is impossible to replace or get rid of the
> > Hibernate
> > > > > key,
> > > > > > is
> > > > > > > > >>> this
> > > > > > > > >>>>>> discussion valid at all?
> > > > > > > > >>>>>>
> > > > > > > > >>>>>
> > > > > > > > >>>>
> > > > > > > > >>>
> > > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Stable binary key representation
Posted by Vladimir Ozerov <vo...@gridgain.com>.
Dima,
Yes, they may explode some internals of our indexes.
06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan" <
dsetrakyan@apache.org> написал:
> Guys,
>
> Isn't the main issue here that we cannot use the Identity Resolvers in
> BTrees in the 2.0 version? If yes, then we have to remove them no matter
> what.
>
> D.
>
> On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <se...@gmail.com>
> wrote:
>
> > Binary key representation is stable when we always have equal serialized
> > bytes when the original keys are equal.
> >
> > Resolver allows you to have some extra info in the Key and equal Keys
> will
> > be serialized into different bytes, which is wrong.
> >
> > Look at the example what you can do with resolvers:
> >
> > We may have some data entry with fields a, b, c. Let's say the unique
> part
> > here is `a` and it the only fields used in Key equals() and hashCode().
> > Still we may have the following layouts:
> >
> > 1. Ka -> Vbc
> > 2. Kab -> Vc
> > 3. Kabc -> Boolean.TRUE
> >
> > The only 1 is a correct layout, others are plain wrong variants (but they
> > are still possible with Resolvers) because everything that does not make
> > Key unique must be in Value.
> >
> > We want to clearly state that if you have something in Key, that is not
> > part of equals(), then the Key is invalid and that stuff must be in
> Value.
> > This allows us to rely on binary representation of a Key to be stable and
> > have some more optimizations and code simplifications with respect to
> these
> > assumptions.
> >
> > Sergi
> >
> > 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
> > valentin.kulichenko@gmail.com>:
> >
> > > Even with my vast expirience I would never claim that I've seen
> > > "everything" :)
> > >
> > > What do you mean by stable binary key representation and how resolvers
> > make
> > > it unstable?
> > >
> > > -Val
> > >
> > > On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
> sergi.vladykin@gmail.com
> > >
> > > wrote:
> > >
> > > > Val,
> > > >
> > > > I know that you have really vast experience in Ignite deployments and
> > > > probably saw everything that can happen. Did you ever see identity
> > > > resolvers use in real life? I guess no.
> > > >
> > > > Hibernate example is bad here, because if their key is unstable
> across
> > > > multiple JVMs, it means that it was not designed for distributed
> > caches a
> > > > priori.
> > > >
> > > > Also knowing in advance about stable binary key representation allows
> > us
> > > to
> > > > apply additional optimizations, like comparing keys without detaching
> > > them
> > > > from offheap memory.
> > > >
> > > > We always will be able to add this stuff back if we see users really
> > need
> > > > it. Let's remove it for 2.0.
> > > >
> > > > Sergi
> > > >
> > > > 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
> > > > valentin.kulichenko@gmail.com>:
> > > >
> > > > > Alex,
> > > > >
> > > > > To be honest, I don't understand the reasoning behind the removal.
> I
> > > > think
> > > > > resolvers provide good flexibility for different corner cases and
> > it's
> > > a
> > > > > good thing to have them. Note that they can be applied not only to
> > > cache
> > > > > keys, but to any binary objects.
> > > > >
> > > > > Hibernate issue is actually a good example of such use case. The
> fact
> > > > that
> > > > > we found an alternative solution doesn't actually mean anything,
> > > because
> > > > > what if this happened not in our module, but in user's application?
> > > > > Unfortunately, we can't predict everything.
> > > > >
> > > > > Error proneness is not a very strong argument either, because in my
> > > view
> > > > > these resolvers are as much error prone as BinaryIdMapper, for
> > example.
> > > > >
> > > > > -Val
> > > > >
> > > > > On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
> > > > > alexey.goncharuk@gmail.com> wrote:
> > > > >
> > > > > > Denis,
> > > > > >
> > > > > > Can you suggest a use-case where identity resolver is needed
> (given
> > > > that
> > > > > we
> > > > > > agree that a key must contain only valuable fields)?
> > > > > >
> > > > > > 2017-04-05 22:08 GMT+03:00 Denis Magda <dm...@apache.org>:
> > > > > >
> > > > > > > Where do you want to remove the identity resolvers from? If
> it’s
> > > > > related
> > > > > > > to the internals of Hibernate module then it’s fine but if you
> > > > suggest
> > > > > > > removing identity resolvers public interfaces then it might be
> a
> > > > haste
> > > > > > > decision.
> > > > > > >
> > > > > > > —
> > > > > > > Denis
> > > > > > >
> > > > > > > > On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
> > > > > > alexey.goncharuk@gmail.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > +1, I see no other reasons to keep it.
> > > > > > > >
> > > > > > > > 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
> > > > sergi.vladykin@gmail.com
> > > > > >:
> > > > > > > >
> > > > > > > >> +1
> > > > > > > >>
> > > > > > > >> Lets drop them.
> > > > > > > >>
> > > > > > > >> Sergi
> > > > > > > >>
> > > > > > > >> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
> > > > > > > >> dmitriy.govorukhin@gmail.com>
> > > > > > > >> :
> > > > > > > >>
> > > > > > > >>> Hi guys, i implemented proxy for IgniteCache in hibernate
> > > > > > integration,
> > > > > > > >> this
> > > > > > > >>> proxy transformate cacheKey to our key wrapper, leaves only
> > > > > required
> > > > > > > >>> field. I think we can remove identity resolve, it should
> not
> > > > broke
> > > > > > > >>> integration with hibernate. Any objections?
> > > > > > > >>>
> > > > > > > >>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin Kulichenko <
> > > > > > > >>> valentin.kulichenko@gmail.com> wrote:
> > > > > > > >>>
> > > > > > > >>>> I'm not saying there is no alternative solution. But let's
> > > > > implement
> > > > > > > it
> > > > > > > >>> and
> > > > > > > >>>> prove that it works first, and remove resolvers only after
> > > that.
> > > > > > > >>>>
> > > > > > > >>>> -Val
> > > > > > > >>>>
> > > > > > > >>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi Vladykin <
> > > > > > > >>> sergi.vladykin@gmail.com
> > > > > > > >>>>>
> > > > > > > >>>> wrote:
> > > > > > > >>>>
> > > > > > > >>>>> Guys, nothing is impossible if you know a bit about
> > > reflection
> > > > in
> > > > > > > >> Java
> > > > > > > >>> :)
> > > > > > > >>>>>
> > > > > > > >>>>> We had a look at the CacheKey class and it is easily
> > > > replaceable.
> > > > > > > >>>>>
> > > > > > > >>>>> Sergi
> > > > > > > >>>>>
> > > > > > > >>>>>
> > > > > > > >>>>>
> > > > > > > >>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
> > > > > > dsetrakyan@apache.org
> > > > > > > >>> :
> > > > > > > >>>>>
> > > > > > > >>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin Kulichenko <
> > > > > > > >>>>>> valentin.kulichenko@gmail.com> wrote:
> > > > > > > >>>>>>
> > > > > > > >>>>>>> "Hibernate key" is the CacheKey class I was referring
> to.
> > > > It's
> > > > > > > >>>> provided
> > > > > > > >>>>>> by
> > > > > > > >>>>>>> Hibernate, not by user and not by us. So I'm not sure
> > it's
> > > > > > > >> possible
> > > > > > > >>>> to
> > > > > > > >>>>>>> replace it.
> > > > > > > >>>>>>>
> > > > > > > >>>>>>
> > > > > > > >>>>>> If it is impossible to replace or get rid of the
> Hibernate
> > > > key,
> > > > > is
> > > > > > > >>> this
> > > > > > > >>>>>> discussion valid at all?
> > > > > > > >>>>>>
> > > > > > > >>>>>
> > > > > > > >>>>
> > > > > > > >>>
> > > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Stable binary key representation
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Guys,
Isn't the main issue here that we cannot use the Identity Resolvers in
BTrees in the 2.0 version? If yes, then we have to remove them no matter
what.
D.
On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <se...@gmail.com>
wrote:
> Binary key representation is stable when we always have equal serialized
> bytes when the original keys are equal.
>
> Resolver allows you to have some extra info in the Key and equal Keys will
> be serialized into different bytes, which is wrong.
>
> Look at the example what you can do with resolvers:
>
> We may have some data entry with fields a, b, c. Let's say the unique part
> here is `a` and it the only fields used in Key equals() and hashCode().
> Still we may have the following layouts:
>
> 1. Ka -> Vbc
> 2. Kab -> Vc
> 3. Kabc -> Boolean.TRUE
>
> The only 1 is a correct layout, others are plain wrong variants (but they
> are still possible with Resolvers) because everything that does not make
> Key unique must be in Value.
>
> We want to clearly state that if you have something in Key, that is not
> part of equals(), then the Key is invalid and that stuff must be in Value.
> This allows us to rely on binary representation of a Key to be stable and
> have some more optimizations and code simplifications with respect to these
> assumptions.
>
> Sergi
>
> 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
> valentin.kulichenko@gmail.com>:
>
> > Even with my vast expirience I would never claim that I've seen
> > "everything" :)
> >
> > What do you mean by stable binary key representation and how resolvers
> make
> > it unstable?
> >
> > -Val
> >
> > On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <sergi.vladykin@gmail.com
> >
> > wrote:
> >
> > > Val,
> > >
> > > I know that you have really vast experience in Ignite deployments and
> > > probably saw everything that can happen. Did you ever see identity
> > > resolvers use in real life? I guess no.
> > >
> > > Hibernate example is bad here, because if their key is unstable across
> > > multiple JVMs, it means that it was not designed for distributed
> caches a
> > > priori.
> > >
> > > Also knowing in advance about stable binary key representation allows
> us
> > to
> > > apply additional optimizations, like comparing keys without detaching
> > them
> > > from offheap memory.
> > >
> > > We always will be able to add this stuff back if we see users really
> need
> > > it. Let's remove it for 2.0.
> > >
> > > Sergi
> > >
> > > 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
> > > valentin.kulichenko@gmail.com>:
> > >
> > > > Alex,
> > > >
> > > > To be honest, I don't understand the reasoning behind the removal. I
> > > think
> > > > resolvers provide good flexibility for different corner cases and
> it's
> > a
> > > > good thing to have them. Note that they can be applied not only to
> > cache
> > > > keys, but to any binary objects.
> > > >
> > > > Hibernate issue is actually a good example of such use case. The fact
> > > that
> > > > we found an alternative solution doesn't actually mean anything,
> > because
> > > > what if this happened not in our module, but in user's application?
> > > > Unfortunately, we can't predict everything.
> > > >
> > > > Error proneness is not a very strong argument either, because in my
> > view
> > > > these resolvers are as much error prone as BinaryIdMapper, for
> example.
> > > >
> > > > -Val
> > > >
> > > > On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
> > > > alexey.goncharuk@gmail.com> wrote:
> > > >
> > > > > Denis,
> > > > >
> > > > > Can you suggest a use-case where identity resolver is needed (given
> > > that
> > > > we
> > > > > agree that a key must contain only valuable fields)?
> > > > >
> > > > > 2017-04-05 22:08 GMT+03:00 Denis Magda <dm...@apache.org>:
> > > > >
> > > > > > Where do you want to remove the identity resolvers from? If it’s
> > > > related
> > > > > > to the internals of Hibernate module then it’s fine but if you
> > > suggest
> > > > > > removing identity resolvers public interfaces then it might be a
> > > haste
> > > > > > decision.
> > > > > >
> > > > > > —
> > > > > > Denis
> > > > > >
> > > > > > > On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
> > > > > alexey.goncharuk@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > +1, I see no other reasons to keep it.
> > > > > > >
> > > > > > > 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
> > > sergi.vladykin@gmail.com
> > > > >:
> > > > > > >
> > > > > > >> +1
> > > > > > >>
> > > > > > >> Lets drop them.
> > > > > > >>
> > > > > > >> Sergi
> > > > > > >>
> > > > > > >> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
> > > > > > >> dmitriy.govorukhin@gmail.com>
> > > > > > >> :
> > > > > > >>
> > > > > > >>> Hi guys, i implemented proxy for IgniteCache in hibernate
> > > > > integration,
> > > > > > >> this
> > > > > > >>> proxy transformate cacheKey to our key wrapper, leaves only
> > > > required
> > > > > > >>> field. I think we can remove identity resolve, it should not
> > > broke
> > > > > > >>> integration with hibernate. Any objections?
> > > > > > >>>
> > > > > > >>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin Kulichenko <
> > > > > > >>> valentin.kulichenko@gmail.com> wrote:
> > > > > > >>>
> > > > > > >>>> I'm not saying there is no alternative solution. But let's
> > > > implement
> > > > > > it
> > > > > > >>> and
> > > > > > >>>> prove that it works first, and remove resolvers only after
> > that.
> > > > > > >>>>
> > > > > > >>>> -Val
> > > > > > >>>>
> > > > > > >>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi Vladykin <
> > > > > > >>> sergi.vladykin@gmail.com
> > > > > > >>>>>
> > > > > > >>>> wrote:
> > > > > > >>>>
> > > > > > >>>>> Guys, nothing is impossible if you know a bit about
> > reflection
> > > in
> > > > > > >> Java
> > > > > > >>> :)
> > > > > > >>>>>
> > > > > > >>>>> We had a look at the CacheKey class and it is easily
> > > replaceable.
> > > > > > >>>>>
> > > > > > >>>>> Sergi
> > > > > > >>>>>
> > > > > > >>>>>
> > > > > > >>>>>
> > > > > > >>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
> > > > > dsetrakyan@apache.org
> > > > > > >>> :
> > > > > > >>>>>
> > > > > > >>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin Kulichenko <
> > > > > > >>>>>> valentin.kulichenko@gmail.com> wrote:
> > > > > > >>>>>>
> > > > > > >>>>>>> "Hibernate key" is the CacheKey class I was referring to.
> > > It's
> > > > > > >>>> provided
> > > > > > >>>>>> by
> > > > > > >>>>>>> Hibernate, not by user and not by us. So I'm not sure
> it's
> > > > > > >> possible
> > > > > > >>>> to
> > > > > > >>>>>>> replace it.
> > > > > > >>>>>>>
> > > > > > >>>>>>
> > > > > > >>>>>> If it is impossible to replace or get rid of the Hibernate
> > > key,
> > > > is
> > > > > > >>> this
> > > > > > >>>>>> discussion valid at all?
> > > > > > >>>>>>
> > > > > > >>>>>
> > > > > > >>>>
> > > > > > >>>
> > > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Stable binary key representation
Posted by Sergi Vladykin <se...@gmail.com>.
Binary key representation is stable when we always have equal serialized
bytes when the original keys are equal.
Resolver allows you to have some extra info in the Key and equal Keys will
be serialized into different bytes, which is wrong.
Look at the example what you can do with resolvers:
We may have some data entry with fields a, b, c. Let's say the unique part
here is `a` and it the only fields used in Key equals() and hashCode().
Still we may have the following layouts:
1. Ka -> Vbc
2. Kab -> Vc
3. Kabc -> Boolean.TRUE
The only 1 is a correct layout, others are plain wrong variants (but they
are still possible with Resolvers) because everything that does not make
Key unique must be in Value.
We want to clearly state that if you have something in Key, that is not
part of equals(), then the Key is invalid and that stuff must be in Value.
This allows us to rely on binary representation of a Key to be stable and
have some more optimizations and code simplifications with respect to these
assumptions.
Sergi
2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
valentin.kulichenko@gmail.com>:
> Even with my vast expirience I would never claim that I've seen
> "everything" :)
>
> What do you mean by stable binary key representation and how resolvers make
> it unstable?
>
> -Val
>
> On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <se...@gmail.com>
> wrote:
>
> > Val,
> >
> > I know that you have really vast experience in Ignite deployments and
> > probably saw everything that can happen. Did you ever see identity
> > resolvers use in real life? I guess no.
> >
> > Hibernate example is bad here, because if their key is unstable across
> > multiple JVMs, it means that it was not designed for distributed caches a
> > priori.
> >
> > Also knowing in advance about stable binary key representation allows us
> to
> > apply additional optimizations, like comparing keys without detaching
> them
> > from offheap memory.
> >
> > We always will be able to add this stuff back if we see users really need
> > it. Let's remove it for 2.0.
> >
> > Sergi
> >
> > 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
> > valentin.kulichenko@gmail.com>:
> >
> > > Alex,
> > >
> > > To be honest, I don't understand the reasoning behind the removal. I
> > think
> > > resolvers provide good flexibility for different corner cases and it's
> a
> > > good thing to have them. Note that they can be applied not only to
> cache
> > > keys, but to any binary objects.
> > >
> > > Hibernate issue is actually a good example of such use case. The fact
> > that
> > > we found an alternative solution doesn't actually mean anything,
> because
> > > what if this happened not in our module, but in user's application?
> > > Unfortunately, we can't predict everything.
> > >
> > > Error proneness is not a very strong argument either, because in my
> view
> > > these resolvers are as much error prone as BinaryIdMapper, for example.
> > >
> > > -Val
> > >
> > > On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
> > > alexey.goncharuk@gmail.com> wrote:
> > >
> > > > Denis,
> > > >
> > > > Can you suggest a use-case where identity resolver is needed (given
> > that
> > > we
> > > > agree that a key must contain only valuable fields)?
> > > >
> > > > 2017-04-05 22:08 GMT+03:00 Denis Magda <dm...@apache.org>:
> > > >
> > > > > Where do you want to remove the identity resolvers from? If it’s
> > > related
> > > > > to the internals of Hibernate module then it’s fine but if you
> > suggest
> > > > > removing identity resolvers public interfaces then it might be a
> > haste
> > > > > decision.
> > > > >
> > > > > —
> > > > > Denis
> > > > >
> > > > > > On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
> > > > alexey.goncharuk@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > +1, I see no other reasons to keep it.
> > > > > >
> > > > > > 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
> > sergi.vladykin@gmail.com
> > > >:
> > > > > >
> > > > > >> +1
> > > > > >>
> > > > > >> Lets drop them.
> > > > > >>
> > > > > >> Sergi
> > > > > >>
> > > > > >> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
> > > > > >> dmitriy.govorukhin@gmail.com>
> > > > > >> :
> > > > > >>
> > > > > >>> Hi guys, i implemented proxy for IgniteCache in hibernate
> > > > integration,
> > > > > >> this
> > > > > >>> proxy transformate cacheKey to our key wrapper, leaves only
> > > required
> > > > > >>> field. I think we can remove identity resolve, it should not
> > broke
> > > > > >>> integration with hibernate. Any objections?
> > > > > >>>
> > > > > >>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin Kulichenko <
> > > > > >>> valentin.kulichenko@gmail.com> wrote:
> > > > > >>>
> > > > > >>>> I'm not saying there is no alternative solution. But let's
> > > implement
> > > > > it
> > > > > >>> and
> > > > > >>>> prove that it works first, and remove resolvers only after
> that.
> > > > > >>>>
> > > > > >>>> -Val
> > > > > >>>>
> > > > > >>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi Vladykin <
> > > > > >>> sergi.vladykin@gmail.com
> > > > > >>>>>
> > > > > >>>> wrote:
> > > > > >>>>
> > > > > >>>>> Guys, nothing is impossible if you know a bit about
> reflection
> > in
> > > > > >> Java
> > > > > >>> :)
> > > > > >>>>>
> > > > > >>>>> We had a look at the CacheKey class and it is easily
> > replaceable.
> > > > > >>>>>
> > > > > >>>>> Sergi
> > > > > >>>>>
> > > > > >>>>>
> > > > > >>>>>
> > > > > >>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
> > > > dsetrakyan@apache.org
> > > > > >>> :
> > > > > >>>>>
> > > > > >>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin Kulichenko <
> > > > > >>>>>> valentin.kulichenko@gmail.com> wrote:
> > > > > >>>>>>
> > > > > >>>>>>> "Hibernate key" is the CacheKey class I was referring to.
> > It's
> > > > > >>>> provided
> > > > > >>>>>> by
> > > > > >>>>>>> Hibernate, not by user and not by us. So I'm not sure it's
> > > > > >> possible
> > > > > >>>> to
> > > > > >>>>>>> replace it.
> > > > > >>>>>>>
> > > > > >>>>>>
> > > > > >>>>>> If it is impossible to replace or get rid of the Hibernate
> > key,
> > > is
> > > > > >>> this
> > > > > >>>>>> discussion valid at all?
> > > > > >>>>>>
> > > > > >>>>>
> > > > > >>>>
> > > > > >>>
> > > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: Stable binary key representation
Posted by Valentin Kulichenko <va...@gmail.com>.
Even with my vast expirience I would never claim that I've seen
"everything" :)
What do you mean by stable binary key representation and how resolvers make
it unstable?
-Val
On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <se...@gmail.com>
wrote:
> Val,
>
> I know that you have really vast experience in Ignite deployments and
> probably saw everything that can happen. Did you ever see identity
> resolvers use in real life? I guess no.
>
> Hibernate example is bad here, because if their key is unstable across
> multiple JVMs, it means that it was not designed for distributed caches a
> priori.
>
> Also knowing in advance about stable binary key representation allows us to
> apply additional optimizations, like comparing keys without detaching them
> from offheap memory.
>
> We always will be able to add this stuff back if we see users really need
> it. Let's remove it for 2.0.
>
> Sergi
>
> 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
> valentin.kulichenko@gmail.com>:
>
> > Alex,
> >
> > To be honest, I don't understand the reasoning behind the removal. I
> think
> > resolvers provide good flexibility for different corner cases and it's a
> > good thing to have them. Note that they can be applied not only to cache
> > keys, but to any binary objects.
> >
> > Hibernate issue is actually a good example of such use case. The fact
> that
> > we found an alternative solution doesn't actually mean anything, because
> > what if this happened not in our module, but in user's application?
> > Unfortunately, we can't predict everything.
> >
> > Error proneness is not a very strong argument either, because in my view
> > these resolvers are as much error prone as BinaryIdMapper, for example.
> >
> > -Val
> >
> > On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com> wrote:
> >
> > > Denis,
> > >
> > > Can you suggest a use-case where identity resolver is needed (given
> that
> > we
> > > agree that a key must contain only valuable fields)?
> > >
> > > 2017-04-05 22:08 GMT+03:00 Denis Magda <dm...@apache.org>:
> > >
> > > > Where do you want to remove the identity resolvers from? If it’s
> > related
> > > > to the internals of Hibernate module then it’s fine but if you
> suggest
> > > > removing identity resolvers public interfaces then it might be a
> haste
> > > > decision.
> > > >
> > > > —
> > > > Denis
> > > >
> > > > > On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
> > > alexey.goncharuk@gmail.com>
> > > > wrote:
> > > > >
> > > > > +1, I see no other reasons to keep it.
> > > > >
> > > > > 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
> sergi.vladykin@gmail.com
> > >:
> > > > >
> > > > >> +1
> > > > >>
> > > > >> Lets drop them.
> > > > >>
> > > > >> Sergi
> > > > >>
> > > > >> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
> > > > >> dmitriy.govorukhin@gmail.com>
> > > > >> :
> > > > >>
> > > > >>> Hi guys, i implemented proxy for IgniteCache in hibernate
> > > integration,
> > > > >> this
> > > > >>> proxy transformate cacheKey to our key wrapper, leaves only
> > required
> > > > >>> field. I think we can remove identity resolve, it should not
> broke
> > > > >>> integration with hibernate. Any objections?
> > > > >>>
> > > > >>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin Kulichenko <
> > > > >>> valentin.kulichenko@gmail.com> wrote:
> > > > >>>
> > > > >>>> I'm not saying there is no alternative solution. But let's
> > implement
> > > > it
> > > > >>> and
> > > > >>>> prove that it works first, and remove resolvers only after that.
> > > > >>>>
> > > > >>>> -Val
> > > > >>>>
> > > > >>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi Vladykin <
> > > > >>> sergi.vladykin@gmail.com
> > > > >>>>>
> > > > >>>> wrote:
> > > > >>>>
> > > > >>>>> Guys, nothing is impossible if you know a bit about reflection
> in
> > > > >> Java
> > > > >>> :)
> > > > >>>>>
> > > > >>>>> We had a look at the CacheKey class and it is easily
> replaceable.
> > > > >>>>>
> > > > >>>>> Sergi
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
> > > dsetrakyan@apache.org
> > > > >>> :
> > > > >>>>>
> > > > >>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin Kulichenko <
> > > > >>>>>> valentin.kulichenko@gmail.com> wrote:
> > > > >>>>>>
> > > > >>>>>>> "Hibernate key" is the CacheKey class I was referring to.
> It's
> > > > >>>> provided
> > > > >>>>>> by
> > > > >>>>>>> Hibernate, not by user and not by us. So I'm not sure it's
> > > > >> possible
> > > > >>>> to
> > > > >>>>>>> replace it.
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>>> If it is impossible to replace or get rid of the Hibernate
> key,
> > is
> > > > >>> this
> > > > >>>>>> discussion valid at all?
> > > > >>>>>>
> > > > >>>>>
> > > > >>>>
> > > > >>>
> > > > >>
> > > >
> > > >
> > >
> >
>
Re: Stable binary key representation
Posted by Sergi Vladykin <se...@gmail.com>.
Val,
I know that you have really vast experience in Ignite deployments and
probably saw everything that can happen. Did you ever see identity
resolvers use in real life? I guess no.
Hibernate example is bad here, because if their key is unstable across
multiple JVMs, it means that it was not designed for distributed caches a
priori.
Also knowing in advance about stable binary key representation allows us to
apply additional optimizations, like comparing keys without detaching them
from offheap memory.
We always will be able to add this stuff back if we see users really need
it. Let's remove it for 2.0.
Sergi
2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
valentin.kulichenko@gmail.com>:
> Alex,
>
> To be honest, I don't understand the reasoning behind the removal. I think
> resolvers provide good flexibility for different corner cases and it's a
> good thing to have them. Note that they can be applied not only to cache
> keys, but to any binary objects.
>
> Hibernate issue is actually a good example of such use case. The fact that
> we found an alternative solution doesn't actually mean anything, because
> what if this happened not in our module, but in user's application?
> Unfortunately, we can't predict everything.
>
> Error proneness is not a very strong argument either, because in my view
> these resolvers are as much error prone as BinaryIdMapper, for example.
>
> -Val
>
> On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
>
> > Denis,
> >
> > Can you suggest a use-case where identity resolver is needed (given that
> we
> > agree that a key must contain only valuable fields)?
> >
> > 2017-04-05 22:08 GMT+03:00 Denis Magda <dm...@apache.org>:
> >
> > > Where do you want to remove the identity resolvers from? If it’s
> related
> > > to the internals of Hibernate module then it’s fine but if you suggest
> > > removing identity resolvers public interfaces then it might be a haste
> > > decision.
> > >
> > > —
> > > Denis
> > >
> > > > On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com>
> > > wrote:
> > > >
> > > > +1, I see no other reasons to keep it.
> > > >
> > > > 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <sergi.vladykin@gmail.com
> >:
> > > >
> > > >> +1
> > > >>
> > > >> Lets drop them.
> > > >>
> > > >> Sergi
> > > >>
> > > >> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
> > > >> dmitriy.govorukhin@gmail.com>
> > > >> :
> > > >>
> > > >>> Hi guys, i implemented proxy for IgniteCache in hibernate
> > integration,
> > > >> this
> > > >>> proxy transformate cacheKey to our key wrapper, leaves only
> required
> > > >>> field. I think we can remove identity resolve, it should not broke
> > > >>> integration with hibernate. Any objections?
> > > >>>
> > > >>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin Kulichenko <
> > > >>> valentin.kulichenko@gmail.com> wrote:
> > > >>>
> > > >>>> I'm not saying there is no alternative solution. But let's
> implement
> > > it
> > > >>> and
> > > >>>> prove that it works first, and remove resolvers only after that.
> > > >>>>
> > > >>>> -Val
> > > >>>>
> > > >>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi Vladykin <
> > > >>> sergi.vladykin@gmail.com
> > > >>>>>
> > > >>>> wrote:
> > > >>>>
> > > >>>>> Guys, nothing is impossible if you know a bit about reflection in
> > > >> Java
> > > >>> :)
> > > >>>>>
> > > >>>>> We had a look at the CacheKey class and it is easily replaceable.
> > > >>>>>
> > > >>>>> Sergi
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
> > dsetrakyan@apache.org
> > > >>> :
> > > >>>>>
> > > >>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin Kulichenko <
> > > >>>>>> valentin.kulichenko@gmail.com> wrote:
> > > >>>>>>
> > > >>>>>>> "Hibernate key" is the CacheKey class I was referring to. It's
> > > >>>> provided
> > > >>>>>> by
> > > >>>>>>> Hibernate, not by user and not by us. So I'm not sure it's
> > > >> possible
> > > >>>> to
> > > >>>>>>> replace it.
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>> If it is impossible to replace or get rid of the Hibernate key,
> is
> > > >>> this
> > > >>>>>> discussion valid at all?
> > > >>>>>>
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> > >
> >
>
Re: Stable binary key representation
Posted by Valentin Kulichenko <va...@gmail.com>.
Alex,
To be honest, I don't understand the reasoning behind the removal. I think
resolvers provide good flexibility for different corner cases and it's a
good thing to have them. Note that they can be applied not only to cache
keys, but to any binary objects.
Hibernate issue is actually a good example of such use case. The fact that
we found an alternative solution doesn't actually mean anything, because
what if this happened not in our module, but in user's application?
Unfortunately, we can't predict everything.
Error proneness is not a very strong argument either, because in my view
these resolvers are as much error prone as BinaryIdMapper, for example.
-Val
On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
alexey.goncharuk@gmail.com> wrote:
> Denis,
>
> Can you suggest a use-case where identity resolver is needed (given that we
> agree that a key must contain only valuable fields)?
>
> 2017-04-05 22:08 GMT+03:00 Denis Magda <dm...@apache.org>:
>
> > Where do you want to remove the identity resolvers from? If it’s related
> > to the internals of Hibernate module then it’s fine but if you suggest
> > removing identity resolvers public interfaces then it might be a haste
> > decision.
> >
> > —
> > Denis
> >
> > > On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
> alexey.goncharuk@gmail.com>
> > wrote:
> > >
> > > +1, I see no other reasons to keep it.
> > >
> > > 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <se...@gmail.com>:
> > >
> > >> +1
> > >>
> > >> Lets drop them.
> > >>
> > >> Sergi
> > >>
> > >> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
> > >> dmitriy.govorukhin@gmail.com>
> > >> :
> > >>
> > >>> Hi guys, i implemented proxy for IgniteCache in hibernate
> integration,
> > >> this
> > >>> proxy transformate cacheKey to our key wrapper, leaves only required
> > >>> field. I think we can remove identity resolve, it should not broke
> > >>> integration with hibernate. Any objections?
> > >>>
> > >>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin Kulichenko <
> > >>> valentin.kulichenko@gmail.com> wrote:
> > >>>
> > >>>> I'm not saying there is no alternative solution. But let's implement
> > it
> > >>> and
> > >>>> prove that it works first, and remove resolvers only after that.
> > >>>>
> > >>>> -Val
> > >>>>
> > >>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi Vladykin <
> > >>> sergi.vladykin@gmail.com
> > >>>>>
> > >>>> wrote:
> > >>>>
> > >>>>> Guys, nothing is impossible if you know a bit about reflection in
> > >> Java
> > >>> :)
> > >>>>>
> > >>>>> We had a look at the CacheKey class and it is easily replaceable.
> > >>>>>
> > >>>>> Sergi
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
> dsetrakyan@apache.org
> > >>> :
> > >>>>>
> > >>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin Kulichenko <
> > >>>>>> valentin.kulichenko@gmail.com> wrote:
> > >>>>>>
> > >>>>>>> "Hibernate key" is the CacheKey class I was referring to. It's
> > >>>> provided
> > >>>>>> by
> > >>>>>>> Hibernate, not by user and not by us. So I'm not sure it's
> > >> possible
> > >>>> to
> > >>>>>>> replace it.
> > >>>>>>>
> > >>>>>>
> > >>>>>> If it is impossible to replace or get rid of the Hibernate key, is
> > >>> this
> > >>>>>> discussion valid at all?
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> >
>
Re: Stable binary key representation
Posted by Alexey Goncharuk <al...@gmail.com>.
Denis,
Can you suggest a use-case where identity resolver is needed (given that we
agree that a key must contain only valuable fields)?
2017-04-05 22:08 GMT+03:00 Denis Magda <dm...@apache.org>:
> Where do you want to remove the identity resolvers from? If it’s related
> to the internals of Hibernate module then it’s fine but if you suggest
> removing identity resolvers public interfaces then it might be a haste
> decision.
>
> —
> Denis
>
> > On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <al...@gmail.com>
> wrote:
> >
> > +1, I see no other reasons to keep it.
> >
> > 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <se...@gmail.com>:
> >
> >> +1
> >>
> >> Lets drop them.
> >>
> >> Sergi
> >>
> >> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
> >> dmitriy.govorukhin@gmail.com>
> >> :
> >>
> >>> Hi guys, i implemented proxy for IgniteCache in hibernate integration,
> >> this
> >>> proxy transformate cacheKey to our key wrapper, leaves only required
> >>> field. I think we can remove identity resolve, it should not broke
> >>> integration with hibernate. Any objections?
> >>>
> >>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin Kulichenko <
> >>> valentin.kulichenko@gmail.com> wrote:
> >>>
> >>>> I'm not saying there is no alternative solution. But let's implement
> it
> >>> and
> >>>> prove that it works first, and remove resolvers only after that.
> >>>>
> >>>> -Val
> >>>>
> >>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi Vladykin <
> >>> sergi.vladykin@gmail.com
> >>>>>
> >>>> wrote:
> >>>>
> >>>>> Guys, nothing is impossible if you know a bit about reflection in
> >> Java
> >>> :)
> >>>>>
> >>>>> We had a look at the CacheKey class and it is easily replaceable.
> >>>>>
> >>>>> Sergi
> >>>>>
> >>>>>
> >>>>>
> >>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org
> >>> :
> >>>>>
> >>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin Kulichenko <
> >>>>>> valentin.kulichenko@gmail.com> wrote:
> >>>>>>
> >>>>>>> "Hibernate key" is the CacheKey class I was referring to. It's
> >>>> provided
> >>>>>> by
> >>>>>>> Hibernate, not by user and not by us. So I'm not sure it's
> >> possible
> >>>> to
> >>>>>>> replace it.
> >>>>>>>
> >>>>>>
> >>>>>> If it is impossible to replace or get rid of the Hibernate key, is
> >>> this
> >>>>>> discussion valid at all?
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>
Re: Stable binary key representation
Posted by Denis Magda <dm...@apache.org>.
Where do you want to remove the identity resolvers from? If it’s related to the internals of Hibernate module then it’s fine but if you suggest removing identity resolvers public interfaces then it might be a haste decision.
—
Denis
> On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <al...@gmail.com> wrote:
>
> +1, I see no other reasons to keep it.
>
> 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <se...@gmail.com>:
>
>> +1
>>
>> Lets drop them.
>>
>> Sergi
>>
>> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
>> dmitriy.govorukhin@gmail.com>
>> :
>>
>>> Hi guys, i implemented proxy for IgniteCache in hibernate integration,
>> this
>>> proxy transformate cacheKey to our key wrapper, leaves only required
>>> field. I think we can remove identity resolve, it should not broke
>>> integration with hibernate. Any objections?
>>>
>>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin Kulichenko <
>>> valentin.kulichenko@gmail.com> wrote:
>>>
>>>> I'm not saying there is no alternative solution. But let's implement it
>>> and
>>>> prove that it works first, and remove resolvers only after that.
>>>>
>>>> -Val
>>>>
>>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi Vladykin <
>>> sergi.vladykin@gmail.com
>>>>>
>>>> wrote:
>>>>
>>>>> Guys, nothing is impossible if you know a bit about reflection in
>> Java
>>> :)
>>>>>
>>>>> We had a look at the CacheKey class and it is easily replaceable.
>>>>>
>>>>> Sergi
>>>>>
>>>>>
>>>>>
>>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org
>>> :
>>>>>
>>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin Kulichenko <
>>>>>> valentin.kulichenko@gmail.com> wrote:
>>>>>>
>>>>>>> "Hibernate key" is the CacheKey class I was referring to. It's
>>>> provided
>>>>>> by
>>>>>>> Hibernate, not by user and not by us. So I'm not sure it's
>> possible
>>>> to
>>>>>>> replace it.
>>>>>>>
>>>>>>
>>>>>> If it is impossible to replace or get rid of the Hibernate key, is
>>> this
>>>>>> discussion valid at all?
>>>>>>
>>>>>
>>>>
>>>
>>
Re: Stable binary key representation
Posted by Alexey Goncharuk <al...@gmail.com>.
+1, I see no other reasons to keep it.
2017-04-05 13:59 GMT+03:00 Sergi Vladykin <se...@gmail.com>:
> +1
>
> Lets drop them.
>
> Sergi
>
> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
> dmitriy.govorukhin@gmail.com>
> :
>
> > Hi guys, i implemented proxy for IgniteCache in hibernate integration,
> this
> > proxy transformate cacheKey to our key wrapper, leaves only required
> > field. I think we can remove identity resolve, it should not broke
> > integration with hibernate. Any objections?
> >
> > On Wed, Mar 29, 2017 at 11:07 PM, Valentin Kulichenko <
> > valentin.kulichenko@gmail.com> wrote:
> >
> > > I'm not saying there is no alternative solution. But let's implement it
> > and
> > > prove that it works first, and remove resolvers only after that.
> > >
> > > -Val
> > >
> > > On Wed, Mar 29, 2017 at 12:18 PM, Sergi Vladykin <
> > sergi.vladykin@gmail.com
> > > >
> > > wrote:
> > >
> > > > Guys, nothing is impossible if you know a bit about reflection in
> Java
> > :)
> > > >
> > > > We had a look at the CacheKey class and it is easily replaceable.
> > > >
> > > > Sergi
> > > >
> > > >
> > > >
> > > > 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org
> >:
> > > >
> > > > > On Wed, Mar 29, 2017 at 11:43 AM, Valentin Kulichenko <
> > > > > valentin.kulichenko@gmail.com> wrote:
> > > > >
> > > > > > "Hibernate key" is the CacheKey class I was referring to. It's
> > > provided
> > > > > by
> > > > > > Hibernate, not by user and not by us. So I'm not sure it's
> possible
> > > to
> > > > > > replace it.
> > > > > >
> > > > >
> > > > > If it is impossible to replace or get rid of the Hibernate key, is
> > this
> > > > > discussion valid at all?
> > > > >
> > > >
> > >
> >
>
Re: Stable binary key representation
Posted by Sergi Vladykin <se...@gmail.com>.
+1
Lets drop them.
Sergi
2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <dm...@gmail.com>
:
> Hi guys, i implemented proxy for IgniteCache in hibernate integration, this
> proxy transformate cacheKey to our key wrapper, leaves only required
> field. I think we can remove identity resolve, it should not broke
> integration with hibernate. Any objections?
>
> On Wed, Mar 29, 2017 at 11:07 PM, Valentin Kulichenko <
> valentin.kulichenko@gmail.com> wrote:
>
> > I'm not saying there is no alternative solution. But let's implement it
> and
> > prove that it works first, and remove resolvers only after that.
> >
> > -Val
> >
> > On Wed, Mar 29, 2017 at 12:18 PM, Sergi Vladykin <
> sergi.vladykin@gmail.com
> > >
> > wrote:
> >
> > > Guys, nothing is impossible if you know a bit about reflection in Java
> :)
> > >
> > > We had a look at the CacheKey class and it is easily replaceable.
> > >
> > > Sergi
> > >
> > >
> > >
> > > 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:
> > >
> > > > On Wed, Mar 29, 2017 at 11:43 AM, Valentin Kulichenko <
> > > > valentin.kulichenko@gmail.com> wrote:
> > > >
> > > > > "Hibernate key" is the CacheKey class I was referring to. It's
> > provided
> > > > by
> > > > > Hibernate, not by user and not by us. So I'm not sure it's possible
> > to
> > > > > replace it.
> > > > >
> > > >
> > > > If it is impossible to replace or get rid of the Hibernate key, is
> this
> > > > discussion valid at all?
> > > >
> > >
> >
>