You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Dmitriy Setrakyan <ds...@apache.org> on 2016/01/21 03:39:56 UTC
supporting collection indexes
Igniters,
I would like us to consider support for indexes into collections (lists,
sets) for object fields. I think we can support it by storing collections
internally in a separate cache and create a special index for it.
For example:
- Create special type of index annotation and config for inner
collections
- Internally, store collection as an additional table with a synthetic
foreign key.
- Have user explicitly do a join between 2 tables when he needs to
select something
The only question I still have is how to handle modifications to
collections. Our current cache access approach would require user to clone
a collection whenever adding or removing elements in it, which can get
quite expensive.
Thoughts?
D.
Re: supporting collection indexes
Posted by Christos Erotocritou <ch...@gridgain.com>.
Ah ok, I get it.
Yes in that sense you are absolutely right, that would be nonsense.
C.
On Fri, Jan 22, 2016 at 2:16 PM, Sergi Vladykin <se...@gmail.com>
wrote:
> Hey Christos!
>
> I've actually agreed with Dmitriy's proposal (which covers the use case
> you've described but with another SQL syntax). It will not require too much
> efforts to implement, will not have too complex config, will have clear
> performance characteristics. This absolutely makes sense.
>
> But for corner cases when for example you don't have Dealer class, but
> instead store collection of Car as a cache value and we should somehow know
> that we need to index the collection... This case looks somewhat artificial
> to me. Of course come rare crazy user may want to do this, but for me it is
> just a wrong domain model and this is what I'm against.
>
> Sergi
>
>
> 2016-01-22 15:29 GMT+03:00 Christos Erotocritou <ch...@gridgain.com>:
>
> > Hey guys,
> >
> > Please bear with me as I am relatively new to Ignite. But here’s my
> > thoughts:
> >
> > I have to agree that we can’t jump in an go build anything that comes to
> > mind but to Sergi’s point about indexing collection objects being
> pointless
> > I would take for instance the example of a car dealership service. The
> > domain model could look something like this:
> >
> > Dealer
> > - name
> > - some_other_field
> > - cars [collection]
> > -- Honda
> > -- Suzuki
> > -- some_other_make
> >
> > If I want to search for a dealer that sells Honda cars for example then
> > being able to index the objects within the collection would be
> benificial.
> >
> > Something like:
> > “ all dealers where cars[*] = ‘Honda’ "
> >
> > Ideally it would be great if we could also index a field within an object
> > in a collection, i.e. checking all users that might have read a specific
> > book:
> > “books[*].id = ’1323421’ "
> >
> > Although the above may need reconsideration of the domain model.
> >
> > Based on my previous experience this is not something that I have seen
> > extensively but definitely something that comes up.
> >
> > Or am I missing the point completely here?
> >
> > Regarding indexing of map values, I’m not exactly sure why you would need
> > that and I do feel that would seem like a bad practice as you should
> split
> > up the object in such cases.
> >
> > Christos.
> >
> >
> > > On 22 Jan 2016, at 10:16, Sergi Vladykin <se...@gmail.com>
> > wrote:
> > >
> > > Yakov,
> > >
> > > I absolutely agree that the idea of trying to support any bullshit that
> > > comes to users mind is wrong and harmful.
> > >
> > > Indexing of a collection when this collection is a cache value looks
> > > useless to me, it is a bad domain model design, we should discourage
> > this.
> > >
> > > Indexing a map cache values is completely different use case from what
> we
> > > discuss here. With respect to our new binary marshaller, will it be
> > > possible to extract value by key from marshalled map? Probably this is
> > > impossible. Thus I think we should discourage stuff like this as well.
> > >
> > > Sergi
> > >
> > > 2016-01-22 12:37 GMT+03:00 Yakov Zhdanov <yz...@apache.org>:
> > >
> > >> I have just noticed another question on dev list - whether it is
> > possible
> > >> to index of objects that are maps' values and maps, in turn, act as
> > cache
> > >> values. This is another point to consider, before taking any decision
> > here.
> > >>
> > >> One more.. What if collection of objects to index act as cache value?
> > >>
> > >> From my standpoint it will be pretty complex to implement this in a
> > >> seamless manner without limitations of any kind.
> > >>
> > >> --Yakov
> > >>
> > >> 2016-01-21 20:27 GMT+03:00 Sergi Vladykin <se...@gmail.com>:
> > >>
> > >>> I think this feature can be implemented.
> > >>>
> > >>> As far as I understand user will not need to clone the collection or
> do
> > >>> anything else from what he does now.
> > >>> We well have to drop all the collection items from that separate
> table
> > >> and
> > >>> add them again on update. Of course we can try to apply some
> heuristics
> > >>> like comparing contents of old collection and new one and add/drop
> > >> changed
> > >>> items.
> > >>>
> > >>> Anyways complex things like collection proxy looks like an overkill
> > here,
> > >>> since as Yakov already pointed out the actual value of this feature
> > will
> > >> be
> > >>> quite limited. Lets keep it simple.
> > >>>
> > >>> Sergi
> > >>>
> > >>> 2016-01-21 13:37 GMT+03:00 Yakov Zhdanov <yz...@apache.org>:
> > >>>
> > >>>> This may be a good feature, but I don't think it will be widely
> used.
> > I
> > >>>> understand that Ignite users want to have their objects stored
> exactly
> > >> in
> > >>>> the format they are used in the application's BL, but in most cases
> I
> > >>> think
> > >>>> that users would benefit if they split they objects.
> > >>>>
> > >>>> For performance considerations I would recommend to store these
> types
> > >>>> separately:
> > >>>> 1. gets/puts to main-type cache will be much faster - no excessive
> > >>>> serializations/deserializations and, most probably, network
> overhead.
> > >>>> 2. dependent collection modification will be the fastest possible
> > >>>> 4. colocation is still here - Ignite can store
> > >>>> 3. collection item modification is seamless (let's say you need to
> > >> modify
> > >>>> "count" of some row of the Order)
> > >>>>
> > >>>> If we choose the way you suggest... Well, this makes me think of
> > >>>> Hibernate-like approaches with sessions, collections mappings and
> > >>> proxies.
> > >>>> I think this will require us to implement collection wrappers and
> > >> proxies
> > >>>> in order to efficiently detect modifications. Btw, we can track such
> > >>>> changes with BinaryObjects methods - BinaryObject may act as a
> proxy.
> > >>>>
> > >>>> Any thoughts?
> > >>>>
> > >>>> --Yakov
> > >>>>
> > >>>> 2016-01-21 5:39 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org
> >:
> > >>>>
> > >>>>> Igniters,
> > >>>>>
> > >>>>> I would like us to consider support for indexes into collections
> > >>> (lists,
> > >>>>> sets) for object fields. I think we can support it by storing
> > >>> collections
> > >>>>> internally in a separate cache and create a special index for it.
> > >>>>>
> > >>>>> For example:
> > >>>>>
> > >>>>> - Create special type of index annotation and config for inner
> > >>>>> collections
> > >>>>> - Internally, store collection as an additional table with a
> > >>> synthetic
> > >>>>> foreign key.
> > >>>>> - Have user explicitly do a join between 2 tables when he needs
> to
> > >>>>> select something
> > >>>>>
> > >>>>> The only question I still have is how to handle modifications to
> > >>>>> collections. Our current cache access approach would require user
> to
> > >>>> clone
> > >>>>> a collection whenever adding or removing elements in it, which can
> > >> get
> > >>>>> quite expensive.
> > >>>>>
> > >>>>> Thoughts?
> > >>>>>
> > >>>>> D.
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> >
>
Re: supporting collection indexes
Posted by Sergi Vladykin <se...@gmail.com>.
Hey Christos!
I've actually agreed with Dmitriy's proposal (which covers the use case
you've described but with another SQL syntax). It will not require too much
efforts to implement, will not have too complex config, will have clear
performance characteristics. This absolutely makes sense.
But for corner cases when for example you don't have Dealer class, but
instead store collection of Car as a cache value and we should somehow know
that we need to index the collection... This case looks somewhat artificial
to me. Of course come rare crazy user may want to do this, but for me it is
just a wrong domain model and this is what I'm against.
Sergi
2016-01-22 15:29 GMT+03:00 Christos Erotocritou <ch...@gridgain.com>:
> Hey guys,
>
> Please bear with me as I am relatively new to Ignite. But here’s my
> thoughts:
>
> I have to agree that we can’t jump in an go build anything that comes to
> mind but to Sergi’s point about indexing collection objects being pointless
> I would take for instance the example of a car dealership service. The
> domain model could look something like this:
>
> Dealer
> - name
> - some_other_field
> - cars [collection]
> -- Honda
> -- Suzuki
> -- some_other_make
>
> If I want to search for a dealer that sells Honda cars for example then
> being able to index the objects within the collection would be benificial.
>
> Something like:
> “ all dealers where cars[*] = ‘Honda’ "
>
> Ideally it would be great if we could also index a field within an object
> in a collection, i.e. checking all users that might have read a specific
> book:
> “books[*].id = ’1323421’ "
>
> Although the above may need reconsideration of the domain model.
>
> Based on my previous experience this is not something that I have seen
> extensively but definitely something that comes up.
>
> Or am I missing the point completely here?
>
> Regarding indexing of map values, I’m not exactly sure why you would need
> that and I do feel that would seem like a bad practice as you should split
> up the object in such cases.
>
> Christos.
>
>
> > On 22 Jan 2016, at 10:16, Sergi Vladykin <se...@gmail.com>
> wrote:
> >
> > Yakov,
> >
> > I absolutely agree that the idea of trying to support any bullshit that
> > comes to users mind is wrong and harmful.
> >
> > Indexing of a collection when this collection is a cache value looks
> > useless to me, it is a bad domain model design, we should discourage
> this.
> >
> > Indexing a map cache values is completely different use case from what we
> > discuss here. With respect to our new binary marshaller, will it be
> > possible to extract value by key from marshalled map? Probably this is
> > impossible. Thus I think we should discourage stuff like this as well.
> >
> > Sergi
> >
> > 2016-01-22 12:37 GMT+03:00 Yakov Zhdanov <yz...@apache.org>:
> >
> >> I have just noticed another question on dev list - whether it is
> possible
> >> to index of objects that are maps' values and maps, in turn, act as
> cache
> >> values. This is another point to consider, before taking any decision
> here.
> >>
> >> One more.. What if collection of objects to index act as cache value?
> >>
> >> From my standpoint it will be pretty complex to implement this in a
> >> seamless manner without limitations of any kind.
> >>
> >> --Yakov
> >>
> >> 2016-01-21 20:27 GMT+03:00 Sergi Vladykin <se...@gmail.com>:
> >>
> >>> I think this feature can be implemented.
> >>>
> >>> As far as I understand user will not need to clone the collection or do
> >>> anything else from what he does now.
> >>> We well have to drop all the collection items from that separate table
> >> and
> >>> add them again on update. Of course we can try to apply some heuristics
> >>> like comparing contents of old collection and new one and add/drop
> >> changed
> >>> items.
> >>>
> >>> Anyways complex things like collection proxy looks like an overkill
> here,
> >>> since as Yakov already pointed out the actual value of this feature
> will
> >> be
> >>> quite limited. Lets keep it simple.
> >>>
> >>> Sergi
> >>>
> >>> 2016-01-21 13:37 GMT+03:00 Yakov Zhdanov <yz...@apache.org>:
> >>>
> >>>> This may be a good feature, but I don't think it will be widely used.
> I
> >>>> understand that Ignite users want to have their objects stored exactly
> >> in
> >>>> the format they are used in the application's BL, but in most cases I
> >>> think
> >>>> that users would benefit if they split they objects.
> >>>>
> >>>> For performance considerations I would recommend to store these types
> >>>> separately:
> >>>> 1. gets/puts to main-type cache will be much faster - no excessive
> >>>> serializations/deserializations and, most probably, network overhead.
> >>>> 2. dependent collection modification will be the fastest possible
> >>>> 4. colocation is still here - Ignite can store
> >>>> 3. collection item modification is seamless (let's say you need to
> >> modify
> >>>> "count" of some row of the Order)
> >>>>
> >>>> If we choose the way you suggest... Well, this makes me think of
> >>>> Hibernate-like approaches with sessions, collections mappings and
> >>> proxies.
> >>>> I think this will require us to implement collection wrappers and
> >> proxies
> >>>> in order to efficiently detect modifications. Btw, we can track such
> >>>> changes with BinaryObjects methods - BinaryObject may act as a proxy.
> >>>>
> >>>> Any thoughts?
> >>>>
> >>>> --Yakov
> >>>>
> >>>> 2016-01-21 5:39 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:
> >>>>
> >>>>> Igniters,
> >>>>>
> >>>>> I would like us to consider support for indexes into collections
> >>> (lists,
> >>>>> sets) for object fields. I think we can support it by storing
> >>> collections
> >>>>> internally in a separate cache and create a special index for it.
> >>>>>
> >>>>> For example:
> >>>>>
> >>>>> - Create special type of index annotation and config for inner
> >>>>> collections
> >>>>> - Internally, store collection as an additional table with a
> >>> synthetic
> >>>>> foreign key.
> >>>>> - Have user explicitly do a join between 2 tables when he needs to
> >>>>> select something
> >>>>>
> >>>>> The only question I still have is how to handle modifications to
> >>>>> collections. Our current cache access approach would require user to
> >>>> clone
> >>>>> a collection whenever adding or removing elements in it, which can
> >> get
> >>>>> quite expensive.
> >>>>>
> >>>>> Thoughts?
> >>>>>
> >>>>> D.
> >>>>>
> >>>>
> >>>
> >>
>
>
Re: supporting collection indexes
Posted by Christos Erotocritou <ch...@gridgain.com>.
Hey guys,
Please bear with me as I am relatively new to Ignite. But here’s my thoughts:
I have to agree that we can’t jump in an go build anything that comes to mind but to Sergi’s point about indexing collection objects being pointless I would take for instance the example of a car dealership service. The domain model could look something like this:
Dealer
- name
- some_other_field
- cars [collection]
-- Honda
-- Suzuki
-- some_other_make
If I want to search for a dealer that sells Honda cars for example then being able to index the objects within the collection would be benificial.
Something like:
“ all dealers where cars[*] = ‘Honda’ "
Ideally it would be great if we could also index a field within an object in a collection, i.e. checking all users that might have read a specific book:
“books[*].id = ’1323421’ "
Although the above may need reconsideration of the domain model.
Based on my previous experience this is not something that I have seen extensively but definitely something that comes up.
Or am I missing the point completely here?
Regarding indexing of map values, I’m not exactly sure why you would need that and I do feel that would seem like a bad practice as you should split up the object in such cases.
Christos.
> On 22 Jan 2016, at 10:16, Sergi Vladykin <se...@gmail.com> wrote:
>
> Yakov,
>
> I absolutely agree that the idea of trying to support any bullshit that
> comes to users mind is wrong and harmful.
>
> Indexing of a collection when this collection is a cache value looks
> useless to me, it is a bad domain model design, we should discourage this.
>
> Indexing a map cache values is completely different use case from what we
> discuss here. With respect to our new binary marshaller, will it be
> possible to extract value by key from marshalled map? Probably this is
> impossible. Thus I think we should discourage stuff like this as well.
>
> Sergi
>
> 2016-01-22 12:37 GMT+03:00 Yakov Zhdanov <yz...@apache.org>:
>
>> I have just noticed another question on dev list - whether it is possible
>> to index of objects that are maps' values and maps, in turn, act as cache
>> values. This is another point to consider, before taking any decision here.
>>
>> One more.. What if collection of objects to index act as cache value?
>>
>> From my standpoint it will be pretty complex to implement this in a
>> seamless manner without limitations of any kind.
>>
>> --Yakov
>>
>> 2016-01-21 20:27 GMT+03:00 Sergi Vladykin <se...@gmail.com>:
>>
>>> I think this feature can be implemented.
>>>
>>> As far as I understand user will not need to clone the collection or do
>>> anything else from what he does now.
>>> We well have to drop all the collection items from that separate table
>> and
>>> add them again on update. Of course we can try to apply some heuristics
>>> like comparing contents of old collection and new one and add/drop
>> changed
>>> items.
>>>
>>> Anyways complex things like collection proxy looks like an overkill here,
>>> since as Yakov already pointed out the actual value of this feature will
>> be
>>> quite limited. Lets keep it simple.
>>>
>>> Sergi
>>>
>>> 2016-01-21 13:37 GMT+03:00 Yakov Zhdanov <yz...@apache.org>:
>>>
>>>> This may be a good feature, but I don't think it will be widely used. I
>>>> understand that Ignite users want to have their objects stored exactly
>> in
>>>> the format they are used in the application's BL, but in most cases I
>>> think
>>>> that users would benefit if they split they objects.
>>>>
>>>> For performance considerations I would recommend to store these types
>>>> separately:
>>>> 1. gets/puts to main-type cache will be much faster - no excessive
>>>> serializations/deserializations and, most probably, network overhead.
>>>> 2. dependent collection modification will be the fastest possible
>>>> 4. colocation is still here - Ignite can store
>>>> 3. collection item modification is seamless (let's say you need to
>> modify
>>>> "count" of some row of the Order)
>>>>
>>>> If we choose the way you suggest... Well, this makes me think of
>>>> Hibernate-like approaches with sessions, collections mappings and
>>> proxies.
>>>> I think this will require us to implement collection wrappers and
>> proxies
>>>> in order to efficiently detect modifications. Btw, we can track such
>>>> changes with BinaryObjects methods - BinaryObject may act as a proxy.
>>>>
>>>> Any thoughts?
>>>>
>>>> --Yakov
>>>>
>>>> 2016-01-21 5:39 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:
>>>>
>>>>> Igniters,
>>>>>
>>>>> I would like us to consider support for indexes into collections
>>> (lists,
>>>>> sets) for object fields. I think we can support it by storing
>>> collections
>>>>> internally in a separate cache and create a special index for it.
>>>>>
>>>>> For example:
>>>>>
>>>>> - Create special type of index annotation and config for inner
>>>>> collections
>>>>> - Internally, store collection as an additional table with a
>>> synthetic
>>>>> foreign key.
>>>>> - Have user explicitly do a join between 2 tables when he needs to
>>>>> select something
>>>>>
>>>>> The only question I still have is how to handle modifications to
>>>>> collections. Our current cache access approach would require user to
>>>> clone
>>>>> a collection whenever adding or removing elements in it, which can
>> get
>>>>> quite expensive.
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> D.
>>>>>
>>>>
>>>
>>
Re: supporting collection indexes
Posted by Sergi Vladykin <se...@gmail.com>.
Yakov,
I absolutely agree that the idea of trying to support any bullshit that
comes to users mind is wrong and harmful.
Indexing of a collection when this collection is a cache value looks
useless to me, it is a bad domain model design, we should discourage this.
Indexing a map cache values is completely different use case from what we
discuss here. With respect to our new binary marshaller, will it be
possible to extract value by key from marshalled map? Probably this is
impossible. Thus I think we should discourage stuff like this as well.
Sergi
2016-01-22 12:37 GMT+03:00 Yakov Zhdanov <yz...@apache.org>:
> I have just noticed another question on dev list - whether it is possible
> to index of objects that are maps' values and maps, in turn, act as cache
> values. This is another point to consider, before taking any decision here.
>
> One more.. What if collection of objects to index act as cache value?
>
> From my standpoint it will be pretty complex to implement this in a
> seamless manner without limitations of any kind.
>
> --Yakov
>
> 2016-01-21 20:27 GMT+03:00 Sergi Vladykin <se...@gmail.com>:
>
> > I think this feature can be implemented.
> >
> > As far as I understand user will not need to clone the collection or do
> > anything else from what he does now.
> > We well have to drop all the collection items from that separate table
> and
> > add them again on update. Of course we can try to apply some heuristics
> > like comparing contents of old collection and new one and add/drop
> changed
> > items.
> >
> > Anyways complex things like collection proxy looks like an overkill here,
> > since as Yakov already pointed out the actual value of this feature will
> be
> > quite limited. Lets keep it simple.
> >
> > Sergi
> >
> > 2016-01-21 13:37 GMT+03:00 Yakov Zhdanov <yz...@apache.org>:
> >
> > > This may be a good feature, but I don't think it will be widely used. I
> > > understand that Ignite users want to have their objects stored exactly
> in
> > > the format they are used in the application's BL, but in most cases I
> > think
> > > that users would benefit if they split they objects.
> > >
> > > For performance considerations I would recommend to store these types
> > > separately:
> > > 1. gets/puts to main-type cache will be much faster - no excessive
> > > serializations/deserializations and, most probably, network overhead.
> > > 2. dependent collection modification will be the fastest possible
> > > 4. colocation is still here - Ignite can store
> > > 3. collection item modification is seamless (let's say you need to
> modify
> > > "count" of some row of the Order)
> > >
> > > If we choose the way you suggest... Well, this makes me think of
> > > Hibernate-like approaches with sessions, collections mappings and
> > proxies.
> > > I think this will require us to implement collection wrappers and
> proxies
> > > in order to efficiently detect modifications. Btw, we can track such
> > > changes with BinaryObjects methods - BinaryObject may act as a proxy.
> > >
> > > Any thoughts?
> > >
> > > --Yakov
> > >
> > > 2016-01-21 5:39 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:
> > >
> > > > Igniters,
> > > >
> > > > I would like us to consider support for indexes into collections
> > (lists,
> > > > sets) for object fields. I think we can support it by storing
> > collections
> > > > internally in a separate cache and create a special index for it.
> > > >
> > > > For example:
> > > >
> > > > - Create special type of index annotation and config for inner
> > > > collections
> > > > - Internally, store collection as an additional table with a
> > synthetic
> > > > foreign key.
> > > > - Have user explicitly do a join between 2 tables when he needs to
> > > > select something
> > > >
> > > > The only question I still have is how to handle modifications to
> > > > collections. Our current cache access approach would require user to
> > > clone
> > > > a collection whenever adding or removing elements in it, which can
> get
> > > > quite expensive.
> > > >
> > > > Thoughts?
> > > >
> > > > D.
> > > >
> > >
> >
>
Re: supporting collection indexes
Posted by Yakov Zhdanov <yz...@apache.org>.
I have just noticed another question on dev list - whether it is possible
to index of objects that are maps' values and maps, in turn, act as cache
values. This is another point to consider, before taking any decision here.
One more.. What if collection of objects to index act as cache value?
>From my standpoint it will be pretty complex to implement this in a
seamless manner without limitations of any kind.
--Yakov
2016-01-21 20:27 GMT+03:00 Sergi Vladykin <se...@gmail.com>:
> I think this feature can be implemented.
>
> As far as I understand user will not need to clone the collection or do
> anything else from what he does now.
> We well have to drop all the collection items from that separate table and
> add them again on update. Of course we can try to apply some heuristics
> like comparing contents of old collection and new one and add/drop changed
> items.
>
> Anyways complex things like collection proxy looks like an overkill here,
> since as Yakov already pointed out the actual value of this feature will be
> quite limited. Lets keep it simple.
>
> Sergi
>
> 2016-01-21 13:37 GMT+03:00 Yakov Zhdanov <yz...@apache.org>:
>
> > This may be a good feature, but I don't think it will be widely used. I
> > understand that Ignite users want to have their objects stored exactly in
> > the format they are used in the application's BL, but in most cases I
> think
> > that users would benefit if they split they objects.
> >
> > For performance considerations I would recommend to store these types
> > separately:
> > 1. gets/puts to main-type cache will be much faster - no excessive
> > serializations/deserializations and, most probably, network overhead.
> > 2. dependent collection modification will be the fastest possible
> > 4. colocation is still here - Ignite can store
> > 3. collection item modification is seamless (let's say you need to modify
> > "count" of some row of the Order)
> >
> > If we choose the way you suggest... Well, this makes me think of
> > Hibernate-like approaches with sessions, collections mappings and
> proxies.
> > I think this will require us to implement collection wrappers and proxies
> > in order to efficiently detect modifications. Btw, we can track such
> > changes with BinaryObjects methods - BinaryObject may act as a proxy.
> >
> > Any thoughts?
> >
> > --Yakov
> >
> > 2016-01-21 5:39 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:
> >
> > > Igniters,
> > >
> > > I would like us to consider support for indexes into collections
> (lists,
> > > sets) for object fields. I think we can support it by storing
> collections
> > > internally in a separate cache and create a special index for it.
> > >
> > > For example:
> > >
> > > - Create special type of index annotation and config for inner
> > > collections
> > > - Internally, store collection as an additional table with a
> synthetic
> > > foreign key.
> > > - Have user explicitly do a join between 2 tables when he needs to
> > > select something
> > >
> > > The only question I still have is how to handle modifications to
> > > collections. Our current cache access approach would require user to
> > clone
> > > a collection whenever adding or removing elements in it, which can get
> > > quite expensive.
> > >
> > > Thoughts?
> > >
> > > D.
> > >
> >
>
Re: supporting collection indexes
Posted by Sergi Vladykin <se...@gmail.com>.
I think this feature can be implemented.
As far as I understand user will not need to clone the collection or do
anything else from what he does now.
We well have to drop all the collection items from that separate table and
add them again on update. Of course we can try to apply some heuristics
like comparing contents of old collection and new one and add/drop changed
items.
Anyways complex things like collection proxy looks like an overkill here,
since as Yakov already pointed out the actual value of this feature will be
quite limited. Lets keep it simple.
Sergi
2016-01-21 13:37 GMT+03:00 Yakov Zhdanov <yz...@apache.org>:
> This may be a good feature, but I don't think it will be widely used. I
> understand that Ignite users want to have their objects stored exactly in
> the format they are used in the application's BL, but in most cases I think
> that users would benefit if they split they objects.
>
> For performance considerations I would recommend to store these types
> separately:
> 1. gets/puts to main-type cache will be much faster - no excessive
> serializations/deserializations and, most probably, network overhead.
> 2. dependent collection modification will be the fastest possible
> 4. colocation is still here - Ignite can store
> 3. collection item modification is seamless (let's say you need to modify
> "count" of some row of the Order)
>
> If we choose the way you suggest... Well, this makes me think of
> Hibernate-like approaches with sessions, collections mappings and proxies.
> I think this will require us to implement collection wrappers and proxies
> in order to efficiently detect modifications. Btw, we can track such
> changes with BinaryObjects methods - BinaryObject may act as a proxy.
>
> Any thoughts?
>
> --Yakov
>
> 2016-01-21 5:39 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:
>
> > Igniters,
> >
> > I would like us to consider support for indexes into collections (lists,
> > sets) for object fields. I think we can support it by storing collections
> > internally in a separate cache and create a special index for it.
> >
> > For example:
> >
> > - Create special type of index annotation and config for inner
> > collections
> > - Internally, store collection as an additional table with a synthetic
> > foreign key.
> > - Have user explicitly do a join between 2 tables when he needs to
> > select something
> >
> > The only question I still have is how to handle modifications to
> > collections. Our current cache access approach would require user to
> clone
> > a collection whenever adding or removing elements in it, which can get
> > quite expensive.
> >
> > Thoughts?
> >
> > D.
> >
>
Re: supporting collection indexes
Posted by Yakov Zhdanov <yz...@apache.org>.
This may be a good feature, but I don't think it will be widely used. I
understand that Ignite users want to have their objects stored exactly in
the format they are used in the application's BL, but in most cases I think
that users would benefit if they split they objects.
For performance considerations I would recommend to store these types
separately:
1. gets/puts to main-type cache will be much faster - no excessive
serializations/deserializations and, most probably, network overhead.
2. dependent collection modification will be the fastest possible
4. colocation is still here - Ignite can store
3. collection item modification is seamless (let's say you need to modify
"count" of some row of the Order)
If we choose the way you suggest... Well, this makes me think of
Hibernate-like approaches with sessions, collections mappings and proxies.
I think this will require us to implement collection wrappers and proxies
in order to efficiently detect modifications. Btw, we can track such
changes with BinaryObjects methods - BinaryObject may act as a proxy.
Any thoughts?
--Yakov
2016-01-21 5:39 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:
> Igniters,
>
> I would like us to consider support for indexes into collections (lists,
> sets) for object fields. I think we can support it by storing collections
> internally in a separate cache and create a special index for it.
>
> For example:
>
> - Create special type of index annotation and config for inner
> collections
> - Internally, store collection as an additional table with a synthetic
> foreign key.
> - Have user explicitly do a join between 2 tables when he needs to
> select something
>
> The only question I still have is how to handle modifications to
> collections. Our current cache access approach would require user to clone
> a collection whenever adding or removing elements in it, which can get
> quite expensive.
>
> Thoughts?
>
> D.
>