You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@geode.apache.org by Jason Huynh <jh...@pivotal.io> on 2017/12/06 00:28:14 UTC
[DISCUSS] Proposal to Deprecate Hash Index
This is a proposal to deprecate existing Hash Index and deprecate the
create hash index apis.
Currently the Hash Index name causes confusion. It is not a traditional
hash look up index, but more of memory savings index. The index does not
store index keys in memory and must hash the keys every time. The index
synchronizes on a backing array and when the backing array needs to be
expanded, it currently needs to rehash all elements in the array. This can
be very problematic for larger data sets.
There were improvements made to one of the functional indexes (compact
range index) prior to open sourcing. These improvements helped reduce the
memory consumption of that index and makes it very similar sized to a hash
index, but the keys still are stored in memory. Probably close enough to
be a replacement for the hash index in most cases. The read/write
performance on it is also faster than the hash index.
If anyone has any objections, please let us know and why.
Thanks,
- Jason
Re: [DISCUSS] Proposal to Deprecate Hash Index
Posted by Bruce Schuchardt <bs...@pivotal.io>.
+1
On 12/6/17 9:41 AM, Barry Oglesby wrote:
> +1
>
> Thanks,
> Barry Oglesby
>
>
> On Tue, Dec 5, 2017 at 9:47 PM, Kirk Lund <kl...@apache.org> wrote:
>
>> +1
>>
>> On Tue, Dec 5, 2017 at 4:34 PM, Dan Smith <ds...@pivotal.io> wrote:
>>
>>> +1
>>>
>>> -Dan
>>>
>>> On Tue, Dec 5, 2017 at 4:28 PM, Jason Huynh <jh...@pivotal.io> wrote:
>>>
>>>> This is a proposal to deprecate existing Hash Index and deprecate the
>>>> create hash index apis.
>>>>
>>>>
>>>> Currently the Hash Index name causes confusion. It is not a traditional
>>>> hash look up index, but more of memory savings index. The index does
>> not
>>>> store index keys in memory and must hash the keys every time. The
>> index
>>>> synchronizes on a backing array and when the backing array needs to be
>>>> expanded, it currently needs to rehash all elements in the array. This
>>> can
>>>> be very problematic for larger data sets.
>>>>
>>>>
>>>> There were improvements made to one of the functional indexes (compact
>>>> range index) prior to open sourcing. These improvements helped reduce
>>> the
>>>> memory consumption of that index and makes it very similar sized to a
>>> hash
>>>> index, but the keys still are stored in memory. Probably close enough
>> to
>>>> be a replacement for the hash index in most cases. The read/write
>>>> performance on it is also faster than the hash index.
>>>>
>>>>
>>>> If anyone has any objections, please let us know and why.
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> - Jason
>>>>
Re: [DISCUSS] Proposal to Deprecate Hash Index
Posted by Barry Oglesby <bo...@pivotal.io>.
+1
Thanks,
Barry Oglesby
On Tue, Dec 5, 2017 at 9:47 PM, Kirk Lund <kl...@apache.org> wrote:
> +1
>
> On Tue, Dec 5, 2017 at 4:34 PM, Dan Smith <ds...@pivotal.io> wrote:
>
> > +1
> >
> > -Dan
> >
> > On Tue, Dec 5, 2017 at 4:28 PM, Jason Huynh <jh...@pivotal.io> wrote:
> >
> > > This is a proposal to deprecate existing Hash Index and deprecate the
> > > create hash index apis.
> > >
> > >
> > > Currently the Hash Index name causes confusion. It is not a traditional
> > > hash look up index, but more of memory savings index. The index does
> not
> > > store index keys in memory and must hash the keys every time. The
> index
> > > synchronizes on a backing array and when the backing array needs to be
> > > expanded, it currently needs to rehash all elements in the array. This
> > can
> > > be very problematic for larger data sets.
> > >
> > >
> > > There were improvements made to one of the functional indexes (compact
> > > range index) prior to open sourcing. These improvements helped reduce
> > the
> > > memory consumption of that index and makes it very similar sized to a
> > hash
> > > index, but the keys still are stored in memory. Probably close enough
> to
> > > be a replacement for the hash index in most cases. The read/write
> > > performance on it is also faster than the hash index.
> > >
> > >
> > > If anyone has any objections, please let us know and why.
> > >
> > >
> > > Thanks,
> > >
> > > - Jason
> > >
> >
>
Re: [DISCUSS] Proposal to Deprecate Hash Index
Posted by Kirk Lund <kl...@apache.org>.
+1
On Tue, Dec 5, 2017 at 4:34 PM, Dan Smith <ds...@pivotal.io> wrote:
> +1
>
> -Dan
>
> On Tue, Dec 5, 2017 at 4:28 PM, Jason Huynh <jh...@pivotal.io> wrote:
>
> > This is a proposal to deprecate existing Hash Index and deprecate the
> > create hash index apis.
> >
> >
> > Currently the Hash Index name causes confusion. It is not a traditional
> > hash look up index, but more of memory savings index. The index does not
> > store index keys in memory and must hash the keys every time. The index
> > synchronizes on a backing array and when the backing array needs to be
> > expanded, it currently needs to rehash all elements in the array. This
> can
> > be very problematic for larger data sets.
> >
> >
> > There were improvements made to one of the functional indexes (compact
> > range index) prior to open sourcing. These improvements helped reduce
> the
> > memory consumption of that index and makes it very similar sized to a
> hash
> > index, but the keys still are stored in memory. Probably close enough to
> > be a replacement for the hash index in most cases. The read/write
> > performance on it is also faster than the hash index.
> >
> >
> > If anyone has any objections, please let us know and why.
> >
> >
> > Thanks,
> >
> > - Jason
> >
>
Re: [DISCUSS] Proposal to Deprecate Hash Index
Posted by Dan Smith <ds...@pivotal.io>.
+1
-Dan
On Tue, Dec 5, 2017 at 4:28 PM, Jason Huynh <jh...@pivotal.io> wrote:
> This is a proposal to deprecate existing Hash Index and deprecate the
> create hash index apis.
>
>
> Currently the Hash Index name causes confusion. It is not a traditional
> hash look up index, but more of memory savings index. The index does not
> store index keys in memory and must hash the keys every time. The index
> synchronizes on a backing array and when the backing array needs to be
> expanded, it currently needs to rehash all elements in the array. This can
> be very problematic for larger data sets.
>
>
> There were improvements made to one of the functional indexes (compact
> range index) prior to open sourcing. These improvements helped reduce the
> memory consumption of that index and makes it very similar sized to a hash
> index, but the keys still are stored in memory. Probably close enough to
> be a replacement for the hash index in most cases. The read/write
> performance on it is also faster than the hash index.
>
>
> If anyone has any objections, please let us know and why.
>
>
> Thanks,
>
> - Jason
>