You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geode.apache.org by Jason Huynh <jh...@pivotal.io> on 2015/11/03 20:21:45 UTC
pdf of slides for Query and Index
Attached are the slides from todays talk.
Please feel free to continue to post questions on the user or dev list and
we will answer them as best as we can.
As a follow up to one of the questions about cluster sizing and indexes
that came up during todays talk:
Although each case will depend on the data itself, here are some things to
help keep in mind when calculating how much memory an index may take. This
does not include any temporary objects/garbage created when executing a
query or created when initially creating the index.
Each index will also have additional overhead for managing the indexes per
region but that size should be very small.
Compact Functional Index:
1.) Reference for every region entry
2.) Size of extracted key
3.) Internal data structures (ConcurrentHashSet) used per index key
Functional Index
1.) Size of copy of value and extracted key per entry to form the tuple
2.) Size of extracted key
3.) Internal data structures per index key, tuple structure
Hash Index
1.) Reference per entry
2.) Internal array size
Map Index
1.) Size of extracted key
2.) Size of internal map
3.) Sizes of Functional or Compact Index
Primary Key
1.) Minor object creation
Thanks,
-Jason
Re: pdf of slides for Query and Index
Posted by Anthony Baker <ab...@pivotal.io>.
Great work Jason!
Side note: OQL syntax can also be used to query a heap dump:
https://visualvm.java.net/oqlhelp.html
Anthony
> On Nov 3, 2015, at 11:38 AM, Gregory Chase <gc...@pivotal.io> wrote:
>
> Thank you Jason for an excellent presentation. We'll post the replay soon as well!
>
> -Greg
>
> On Tue, Nov 3, 2015 at 11:36 AM, Jason Huynh <jhuynh@pivotal.io <ma...@pivotal.io>> wrote:
> Here are the slides on slideshare
>
> http://www.slideshare.net/huynhja/oql-querying-and-indexes <http://www.slideshare.net/huynhja/oql-querying-and-indexes>
>
> On Tue, Nov 3, 2015 at 11:24 AM, Gregory Chase <gchase@pivotal.io <ma...@pivotal.io>> wrote:
> Attachment didn't come through. Please post on Slideshare.
>
> Alternatively feel free to post editable form of the PPT to here:
> https://github.com/Pivotal-Open-Source-Hub/POSH-Talks <https://github.com/Pivotal-Open-Source-Hub/POSH-Talks>
>
> -Greg
>
> On Tue, Nov 3, 2015 at 11:21 AM, Jason Huynh <jhuynh@pivotal.io <ma...@pivotal.io>> wrote:
>
> > Attached are the slides from todays talk.
> >
> > Please feel free to continue to post questions on the user or dev list and
> > we will answer them as best as we can.
> >
> > As a follow up to one of the questions about cluster sizing and indexes
> > that came up during todays talk:
> > Although each case will depend on the data itself, here are some things to
> > help keep in mind when calculating how much memory an index may take. This
> > does not include any temporary objects/garbage created when executing a
> > query or created when initially creating the index.
> > Each index will also have additional overhead for managing the indexes per
> > region but that size should be very small.
> >
> > Compact Functional Index:
> > 1.) Reference for every region entry
> > 2.) Size of extracted key
> > 3.) Internal data structures (ConcurrentHashSet) used per index key
> >
> > Functional Index
> > 1.) Size of copy of value and extracted key per entry to form the tuple
> > 2.) Size of extracted key
> > 3.) Internal data structures per index key, tuple structure
> >
> > Hash Index
> > 1.) Reference per entry
> > 2.) Internal array size
> >
> > Map Index
> > 1.) Size of extracted key
> > 2.) Size of internal map
> > 3.) Sizes of Functional or Compact Index
> >
> > Primary Key
> > 1.) Minor object creation
> >
> >
> > Thanks,
> > -Jason
> >
> >
> >
> >
>
>
> --
> Greg Chase
>
> Director of Big Data Communities
> http://www.pivotal.io/big-data <http://www.pivotal.io/big-data>
>
> Pivotal Software
> http://www.pivotal.io/ <http://www.pivotal.io/>
>
> 650-215-0477 <tel:650-215-0477>
> @GregChase
> Blog: http://geekmarketing.biz/ <http://geekmarketing.biz/>
>
>
>
>
> --
> Greg Chase
>
> Director of Big Data Communities
> http://www.pivotal.io/big-data <http://www.pivotal.io/big-data>
>
> Pivotal Software
> http://www.pivotal.io/ <http://www.pivotal.io/>
>
> 650-215-0477
> @GregChase
> Blog: http://geekmarketing.biz/ <http://geekmarketing.biz/>
>
Re: pdf of slides for Query and Index
Posted by Anthony Baker <ab...@pivotal.io>.
Great work Jason!
Side note: OQL syntax can also be used to query a heap dump:
https://visualvm.java.net/oqlhelp.html
Anthony
> On Nov 3, 2015, at 11:38 AM, Gregory Chase <gc...@pivotal.io> wrote:
>
> Thank you Jason for an excellent presentation. We'll post the replay soon as well!
>
> -Greg
>
> On Tue, Nov 3, 2015 at 11:36 AM, Jason Huynh <jhuynh@pivotal.io <ma...@pivotal.io>> wrote:
> Here are the slides on slideshare
>
> http://www.slideshare.net/huynhja/oql-querying-and-indexes <http://www.slideshare.net/huynhja/oql-querying-and-indexes>
>
> On Tue, Nov 3, 2015 at 11:24 AM, Gregory Chase <gchase@pivotal.io <ma...@pivotal.io>> wrote:
> Attachment didn't come through. Please post on Slideshare.
>
> Alternatively feel free to post editable form of the PPT to here:
> https://github.com/Pivotal-Open-Source-Hub/POSH-Talks <https://github.com/Pivotal-Open-Source-Hub/POSH-Talks>
>
> -Greg
>
> On Tue, Nov 3, 2015 at 11:21 AM, Jason Huynh <jhuynh@pivotal.io <ma...@pivotal.io>> wrote:
>
> > Attached are the slides from todays talk.
> >
> > Please feel free to continue to post questions on the user or dev list and
> > we will answer them as best as we can.
> >
> > As a follow up to one of the questions about cluster sizing and indexes
> > that came up during todays talk:
> > Although each case will depend on the data itself, here are some things to
> > help keep in mind when calculating how much memory an index may take. This
> > does not include any temporary objects/garbage created when executing a
> > query or created when initially creating the index.
> > Each index will also have additional overhead for managing the indexes per
> > region but that size should be very small.
> >
> > Compact Functional Index:
> > 1.) Reference for every region entry
> > 2.) Size of extracted key
> > 3.) Internal data structures (ConcurrentHashSet) used per index key
> >
> > Functional Index
> > 1.) Size of copy of value and extracted key per entry to form the tuple
> > 2.) Size of extracted key
> > 3.) Internal data structures per index key, tuple structure
> >
> > Hash Index
> > 1.) Reference per entry
> > 2.) Internal array size
> >
> > Map Index
> > 1.) Size of extracted key
> > 2.) Size of internal map
> > 3.) Sizes of Functional or Compact Index
> >
> > Primary Key
> > 1.) Minor object creation
> >
> >
> > Thanks,
> > -Jason
> >
> >
> >
> >
>
>
> --
> Greg Chase
>
> Director of Big Data Communities
> http://www.pivotal.io/big-data <http://www.pivotal.io/big-data>
>
> Pivotal Software
> http://www.pivotal.io/ <http://www.pivotal.io/>
>
> 650-215-0477 <tel:650-215-0477>
> @GregChase
> Blog: http://geekmarketing.biz/ <http://geekmarketing.biz/>
>
>
>
>
> --
> Greg Chase
>
> Director of Big Data Communities
> http://www.pivotal.io/big-data <http://www.pivotal.io/big-data>
>
> Pivotal Software
> http://www.pivotal.io/ <http://www.pivotal.io/>
>
> 650-215-0477
> @GregChase
> Blog: http://geekmarketing.biz/ <http://geekmarketing.biz/>
>
Re: pdf of slides for Query and Index
Posted by Jason Huynh <jh...@pivotal.io>.
Hi Wes,
I think Greg or Amy are the ones that put the videos together. I am not
sure about the status...
-Jason
On Wed, Jan 13, 2016 at 10:35 AM, Wes Williams <ww...@pivotal.io> wrote:
> Hi Jason,
>
> do you have the replay of your presentation yet?
>
> Thanks
>
> *Wes Williams | Pivotal Sr. **Data Engineer*
> 781.606.0325
> http://pivotal.io/big-data/pivotal-gemfire
>
> On Sun, Nov 8, 2015 at 6:22 PM, Greg Chase <gc...@pivotal.io> wrote:
>
>> Will post when it's available
>>
>> This email encrypted by tiny buttons & fat thumbs, beta voice
>> recognition, and autocorrect on my iPhone.
>>
>> On Nov 8, 2015, at 3:13 PM, Wes Williams <ww...@pivotal.io> wrote:
>>
>> I'm looking for the replay. Any idea when we can watch this?
>>
>> Thanks,
>>
>> *Wes Williams | Pivotal Sr. **Data Engineer*
>> 781.606.0325
>> http://pivotal.io/big-data/pivotal-gemfire
>>
>> On Tue, Nov 3, 2015 at 2:38 PM, Gregory Chase <gc...@pivotal.io> wrote:
>>
>>> Thank you Jason for an excellent presentation. We'll post the replay soon
>>> as well!
>>>
>>> -Greg
>>>
>>> On Tue, Nov 3, 2015 at 11:36 AM, Jason Huynh <jh...@pivotal.io> wrote:
>>>
>>> > Here are the slides on slideshare
>>> >
>>> > http://www.slideshare.net/huynhja/oql-querying-and-indexes
>>> >
>>> > On Tue, Nov 3, 2015 at 11:24 AM, Gregory Chase <gc...@pivotal.io>
>>> wrote:
>>> >
>>> >> Attachment didn't come through. Please post on Slideshare.
>>> >>
>>> >> Alternatively feel free to post editable form of the PPT to here:
>>> >> https://github.com/Pivotal-Open-Source-Hub/POSH-Talks
>>> >>
>>> >> -Greg
>>> >>
>>> >> On Tue, Nov 3, 2015 at 11:21 AM, Jason Huynh <jh...@pivotal.io>
>>> wrote:
>>> >>
>>> >> > Attached are the slides from todays talk.
>>> >> >
>>> >> > Please feel free to continue to post questions on the user or dev
>>> list
>>> >> and
>>> >> > we will answer them as best as we can.
>>> >> >
>>> >> > As a follow up to one of the questions about cluster sizing and
>>> indexes
>>> >> > that came up during todays talk:
>>> >> > Although each case will depend on the data itself, here are some
>>> things
>>> >> to
>>> >> > help keep in mind when calculating how much memory an index may
>>> take.
>>> >> This
>>> >> > does not include any temporary objects/garbage created when
>>> executing a
>>> >> > query or created when initially creating the index.
>>> >> > Each index will also have additional overhead for managing the
>>> indexes
>>> >> per
>>> >> > region but that size should be very small.
>>> >> >
>>> >> > Compact Functional Index:
>>> >> > 1.) Reference for every region entry
>>> >> > 2.) Size of extracted key
>>> >> > 3.) Internal data structures (ConcurrentHashSet) used per index key
>>> >> >
>>> >> > Functional Index
>>> >> > 1.) Size of copy of value and extracted key per entry to form the
>>> tuple
>>> >> > 2.) Size of extracted key
>>> >> > 3.) Internal data structures per index key, tuple structure
>>> >> >
>>> >> > Hash Index
>>> >> > 1.) Reference per entry
>>> >> > 2.) Internal array size
>>> >> >
>>> >> > Map Index
>>> >> > 1.) Size of extracted key
>>> >> > 2.) Size of internal map
>>> >> > 3.) Sizes of Functional or Compact Index
>>> >> >
>>> >> > Primary Key
>>> >> > 1.) Minor object creation
>>> >> >
>>> >> >
>>> >> > Thanks,
>>> >> > -Jason
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >>
>>> >>
>>> >> --
>>> >> Greg Chase
>>> >>
>>> >> Director of Big Data Communities
>>> >> http://www.pivotal.io/big-data
>>> >>
>>> >> Pivotal Software
>>> >> http://www.pivotal.io/
>>> >>
>>> >> 650-215-0477
>>> >> @GregChase
>>> >> Blog: http://geekmarketing.biz/
>>> >>
>>> >
>>> >
>>>
>>>
>>> --
>>> Greg Chase
>>>
>>> Director of Big Data Communities
>>> http://www.pivotal.io/big-data
>>>
>>> Pivotal Software
>>> http://www.pivotal.io/
>>>
>>> 650-215-0477
>>> @GregChase
>>> Blog: http://geekmarketing.biz/
>>>
>>
>>
>
Re: pdf of slides for Query and Index
Posted by Wes Williams <ww...@pivotal.io>.
Hi Jason,
do you have the replay of your presentation yet?
Thanks
*Wes Williams | Pivotal Sr. **Data Engineer*
781.606.0325
http://pivotal.io/big-data/pivotal-gemfire
On Sun, Nov 8, 2015 at 6:22 PM, Greg Chase <gc...@pivotal.io> wrote:
> Will post when it's available
>
> This email encrypted by tiny buttons & fat thumbs, beta voice recognition,
> and autocorrect on my iPhone.
>
> On Nov 8, 2015, at 3:13 PM, Wes Williams <ww...@pivotal.io> wrote:
>
> I'm looking for the replay. Any idea when we can watch this?
>
> Thanks,
>
> *Wes Williams | Pivotal Sr. **Data Engineer*
> 781.606.0325
> http://pivotal.io/big-data/pivotal-gemfire
>
> On Tue, Nov 3, 2015 at 2:38 PM, Gregory Chase <gc...@pivotal.io> wrote:
>
>> Thank you Jason for an excellent presentation. We'll post the replay soon
>> as well!
>>
>> -Greg
>>
>> On Tue, Nov 3, 2015 at 11:36 AM, Jason Huynh <jh...@pivotal.io> wrote:
>>
>> > Here are the slides on slideshare
>> >
>> > http://www.slideshare.net/huynhja/oql-querying-and-indexes
>> >
>> > On Tue, Nov 3, 2015 at 11:24 AM, Gregory Chase <gc...@pivotal.io>
>> wrote:
>> >
>> >> Attachment didn't come through. Please post on Slideshare.
>> >>
>> >> Alternatively feel free to post editable form of the PPT to here:
>> >> https://github.com/Pivotal-Open-Source-Hub/POSH-Talks
>> >>
>> >> -Greg
>> >>
>> >> On Tue, Nov 3, 2015 at 11:21 AM, Jason Huynh <jh...@pivotal.io>
>> wrote:
>> >>
>> >> > Attached are the slides from todays talk.
>> >> >
>> >> > Please feel free to continue to post questions on the user or dev
>> list
>> >> and
>> >> > we will answer them as best as we can.
>> >> >
>> >> > As a follow up to one of the questions about cluster sizing and
>> indexes
>> >> > that came up during todays talk:
>> >> > Although each case will depend on the data itself, here are some
>> things
>> >> to
>> >> > help keep in mind when calculating how much memory an index may take.
>> >> This
>> >> > does not include any temporary objects/garbage created when
>> executing a
>> >> > query or created when initially creating the index.
>> >> > Each index will also have additional overhead for managing the
>> indexes
>> >> per
>> >> > region but that size should be very small.
>> >> >
>> >> > Compact Functional Index:
>> >> > 1.) Reference for every region entry
>> >> > 2.) Size of extracted key
>> >> > 3.) Internal data structures (ConcurrentHashSet) used per index key
>> >> >
>> >> > Functional Index
>> >> > 1.) Size of copy of value and extracted key per entry to form the
>> tuple
>> >> > 2.) Size of extracted key
>> >> > 3.) Internal data structures per index key, tuple structure
>> >> >
>> >> > Hash Index
>> >> > 1.) Reference per entry
>> >> > 2.) Internal array size
>> >> >
>> >> > Map Index
>> >> > 1.) Size of extracted key
>> >> > 2.) Size of internal map
>> >> > 3.) Sizes of Functional or Compact Index
>> >> >
>> >> > Primary Key
>> >> > 1.) Minor object creation
>> >> >
>> >> >
>> >> > Thanks,
>> >> > -Jason
>> >> >
>> >> >
>> >> >
>> >> >
>> >>
>> >>
>> >> --
>> >> Greg Chase
>> >>
>> >> Director of Big Data Communities
>> >> http://www.pivotal.io/big-data
>> >>
>> >> Pivotal Software
>> >> http://www.pivotal.io/
>> >>
>> >> 650-215-0477
>> >> @GregChase
>> >> Blog: http://geekmarketing.biz/
>> >>
>> >
>> >
>>
>>
>> --
>> Greg Chase
>>
>> Director of Big Data Communities
>> http://www.pivotal.io/big-data
>>
>> Pivotal Software
>> http://www.pivotal.io/
>>
>> 650-215-0477
>> @GregChase
>> Blog: http://geekmarketing.biz/
>>
>
>
Re: pdf of slides for Query and Index
Posted by Gregory Chase <gc...@pivotal.io>.
Thank you Jason for an excellent presentation. We'll post the replay soon
as well!
-Greg
On Tue, Nov 3, 2015 at 11:36 AM, Jason Huynh <jh...@pivotal.io> wrote:
> Here are the slides on slideshare
>
> http://www.slideshare.net/huynhja/oql-querying-and-indexes
>
> On Tue, Nov 3, 2015 at 11:24 AM, Gregory Chase <gc...@pivotal.io> wrote:
>
>> Attachment didn't come through. Please post on Slideshare.
>>
>> Alternatively feel free to post editable form of the PPT to here:
>> https://github.com/Pivotal-Open-Source-Hub/POSH-Talks
>>
>> -Greg
>>
>> On Tue, Nov 3, 2015 at 11:21 AM, Jason Huynh <jh...@pivotal.io> wrote:
>>
>> > Attached are the slides from todays talk.
>> >
>> > Please feel free to continue to post questions on the user or dev list
>> and
>> > we will answer them as best as we can.
>> >
>> > As a follow up to one of the questions about cluster sizing and indexes
>> > that came up during todays talk:
>> > Although each case will depend on the data itself, here are some things
>> to
>> > help keep in mind when calculating how much memory an index may take.
>> This
>> > does not include any temporary objects/garbage created when executing a
>> > query or created when initially creating the index.
>> > Each index will also have additional overhead for managing the indexes
>> per
>> > region but that size should be very small.
>> >
>> > Compact Functional Index:
>> > 1.) Reference for every region entry
>> > 2.) Size of extracted key
>> > 3.) Internal data structures (ConcurrentHashSet) used per index key
>> >
>> > Functional Index
>> > 1.) Size of copy of value and extracted key per entry to form the tuple
>> > 2.) Size of extracted key
>> > 3.) Internal data structures per index key, tuple structure
>> >
>> > Hash Index
>> > 1.) Reference per entry
>> > 2.) Internal array size
>> >
>> > Map Index
>> > 1.) Size of extracted key
>> > 2.) Size of internal map
>> > 3.) Sizes of Functional or Compact Index
>> >
>> > Primary Key
>> > 1.) Minor object creation
>> >
>> >
>> > Thanks,
>> > -Jason
>> >
>> >
>> >
>> >
>>
>>
>> --
>> Greg Chase
>>
>> Director of Big Data Communities
>> http://www.pivotal.io/big-data
>>
>> Pivotal Software
>> http://www.pivotal.io/
>>
>> 650-215-0477
>> @GregChase
>> Blog: http://geekmarketing.biz/
>>
>
>
--
Greg Chase
Director of Big Data Communities
http://www.pivotal.io/big-data
Pivotal Software
http://www.pivotal.io/
650-215-0477
@GregChase
Blog: http://geekmarketing.biz/
Re: pdf of slides for Query and Index
Posted by Gregory Chase <gc...@pivotal.io>.
Thank you Jason for an excellent presentation. We'll post the replay soon
as well!
-Greg
On Tue, Nov 3, 2015 at 11:36 AM, Jason Huynh <jh...@pivotal.io> wrote:
> Here are the slides on slideshare
>
> http://www.slideshare.net/huynhja/oql-querying-and-indexes
>
> On Tue, Nov 3, 2015 at 11:24 AM, Gregory Chase <gc...@pivotal.io> wrote:
>
>> Attachment didn't come through. Please post on Slideshare.
>>
>> Alternatively feel free to post editable form of the PPT to here:
>> https://github.com/Pivotal-Open-Source-Hub/POSH-Talks
>>
>> -Greg
>>
>> On Tue, Nov 3, 2015 at 11:21 AM, Jason Huynh <jh...@pivotal.io> wrote:
>>
>> > Attached are the slides from todays talk.
>> >
>> > Please feel free to continue to post questions on the user or dev list
>> and
>> > we will answer them as best as we can.
>> >
>> > As a follow up to one of the questions about cluster sizing and indexes
>> > that came up during todays talk:
>> > Although each case will depend on the data itself, here are some things
>> to
>> > help keep in mind when calculating how much memory an index may take.
>> This
>> > does not include any temporary objects/garbage created when executing a
>> > query or created when initially creating the index.
>> > Each index will also have additional overhead for managing the indexes
>> per
>> > region but that size should be very small.
>> >
>> > Compact Functional Index:
>> > 1.) Reference for every region entry
>> > 2.) Size of extracted key
>> > 3.) Internal data structures (ConcurrentHashSet) used per index key
>> >
>> > Functional Index
>> > 1.) Size of copy of value and extracted key per entry to form the tuple
>> > 2.) Size of extracted key
>> > 3.) Internal data structures per index key, tuple structure
>> >
>> > Hash Index
>> > 1.) Reference per entry
>> > 2.) Internal array size
>> >
>> > Map Index
>> > 1.) Size of extracted key
>> > 2.) Size of internal map
>> > 3.) Sizes of Functional or Compact Index
>> >
>> > Primary Key
>> > 1.) Minor object creation
>> >
>> >
>> > Thanks,
>> > -Jason
>> >
>> >
>> >
>> >
>>
>>
>> --
>> Greg Chase
>>
>> Director of Big Data Communities
>> http://www.pivotal.io/big-data
>>
>> Pivotal Software
>> http://www.pivotal.io/
>>
>> 650-215-0477
>> @GregChase
>> Blog: http://geekmarketing.biz/
>>
>
>
--
Greg Chase
Director of Big Data Communities
http://www.pivotal.io/big-data
Pivotal Software
http://www.pivotal.io/
650-215-0477
@GregChase
Blog: http://geekmarketing.biz/
Re: pdf of slides for Query and Index
Posted by Jason Huynh <jh...@pivotal.io>.
Here are the slides on slideshare
http://www.slideshare.net/huynhja/oql-querying-and-indexes
On Tue, Nov 3, 2015 at 11:24 AM, Gregory Chase <gc...@pivotal.io> wrote:
> Attachment didn't come through. Please post on Slideshare.
>
> Alternatively feel free to post editable form of the PPT to here:
> https://github.com/Pivotal-Open-Source-Hub/POSH-Talks
>
> -Greg
>
> On Tue, Nov 3, 2015 at 11:21 AM, Jason Huynh <jh...@pivotal.io> wrote:
>
> > Attached are the slides from todays talk.
> >
> > Please feel free to continue to post questions on the user or dev list
> and
> > we will answer them as best as we can.
> >
> > As a follow up to one of the questions about cluster sizing and indexes
> > that came up during todays talk:
> > Although each case will depend on the data itself, here are some things
> to
> > help keep in mind when calculating how much memory an index may take.
> This
> > does not include any temporary objects/garbage created when executing a
> > query or created when initially creating the index.
> > Each index will also have additional overhead for managing the indexes
> per
> > region but that size should be very small.
> >
> > Compact Functional Index:
> > 1.) Reference for every region entry
> > 2.) Size of extracted key
> > 3.) Internal data structures (ConcurrentHashSet) used per index key
> >
> > Functional Index
> > 1.) Size of copy of value and extracted key per entry to form the tuple
> > 2.) Size of extracted key
> > 3.) Internal data structures per index key, tuple structure
> >
> > Hash Index
> > 1.) Reference per entry
> > 2.) Internal array size
> >
> > Map Index
> > 1.) Size of extracted key
> > 2.) Size of internal map
> > 3.) Sizes of Functional or Compact Index
> >
> > Primary Key
> > 1.) Minor object creation
> >
> >
> > Thanks,
> > -Jason
> >
> >
> >
> >
>
>
> --
> Greg Chase
>
> Director of Big Data Communities
> http://www.pivotal.io/big-data
>
> Pivotal Software
> http://www.pivotal.io/
>
> 650-215-0477
> @GregChase
> Blog: http://geekmarketing.biz/
>
Re: pdf of slides for Query and Index
Posted by Jason Huynh <jh...@pivotal.io>.
Here are the slides on slideshare
http://www.slideshare.net/huynhja/oql-querying-and-indexes
On Tue, Nov 3, 2015 at 11:24 AM, Gregory Chase <gc...@pivotal.io> wrote:
> Attachment didn't come through. Please post on Slideshare.
>
> Alternatively feel free to post editable form of the PPT to here:
> https://github.com/Pivotal-Open-Source-Hub/POSH-Talks
>
> -Greg
>
> On Tue, Nov 3, 2015 at 11:21 AM, Jason Huynh <jh...@pivotal.io> wrote:
>
> > Attached are the slides from todays talk.
> >
> > Please feel free to continue to post questions on the user or dev list
> and
> > we will answer them as best as we can.
> >
> > As a follow up to one of the questions about cluster sizing and indexes
> > that came up during todays talk:
> > Although each case will depend on the data itself, here are some things
> to
> > help keep in mind when calculating how much memory an index may take.
> This
> > does not include any temporary objects/garbage created when executing a
> > query or created when initially creating the index.
> > Each index will also have additional overhead for managing the indexes
> per
> > region but that size should be very small.
> >
> > Compact Functional Index:
> > 1.) Reference for every region entry
> > 2.) Size of extracted key
> > 3.) Internal data structures (ConcurrentHashSet) used per index key
> >
> > Functional Index
> > 1.) Size of copy of value and extracted key per entry to form the tuple
> > 2.) Size of extracted key
> > 3.) Internal data structures per index key, tuple structure
> >
> > Hash Index
> > 1.) Reference per entry
> > 2.) Internal array size
> >
> > Map Index
> > 1.) Size of extracted key
> > 2.) Size of internal map
> > 3.) Sizes of Functional or Compact Index
> >
> > Primary Key
> > 1.) Minor object creation
> >
> >
> > Thanks,
> > -Jason
> >
> >
> >
> >
>
>
> --
> Greg Chase
>
> Director of Big Data Communities
> http://www.pivotal.io/big-data
>
> Pivotal Software
> http://www.pivotal.io/
>
> 650-215-0477
> @GregChase
> Blog: http://geekmarketing.biz/
>
Re: pdf of slides for Query and Index
Posted by Gregory Chase <gc...@pivotal.io>.
Attachment didn't come through. Please post on Slideshare.
Alternatively feel free to post editable form of the PPT to here:
https://github.com/Pivotal-Open-Source-Hub/POSH-Talks
-Greg
On Tue, Nov 3, 2015 at 11:21 AM, Jason Huynh <jh...@pivotal.io> wrote:
> Attached are the slides from todays talk.
>
> Please feel free to continue to post questions on the user or dev list and
> we will answer them as best as we can.
>
> As a follow up to one of the questions about cluster sizing and indexes
> that came up during todays talk:
> Although each case will depend on the data itself, here are some things to
> help keep in mind when calculating how much memory an index may take. This
> does not include any temporary objects/garbage created when executing a
> query or created when initially creating the index.
> Each index will also have additional overhead for managing the indexes per
> region but that size should be very small.
>
> Compact Functional Index:
> 1.) Reference for every region entry
> 2.) Size of extracted key
> 3.) Internal data structures (ConcurrentHashSet) used per index key
>
> Functional Index
> 1.) Size of copy of value and extracted key per entry to form the tuple
> 2.) Size of extracted key
> 3.) Internal data structures per index key, tuple structure
>
> Hash Index
> 1.) Reference per entry
> 2.) Internal array size
>
> Map Index
> 1.) Size of extracted key
> 2.) Size of internal map
> 3.) Sizes of Functional or Compact Index
>
> Primary Key
> 1.) Minor object creation
>
>
> Thanks,
> -Jason
>
>
>
>
--
Greg Chase
Director of Big Data Communities
http://www.pivotal.io/big-data
Pivotal Software
http://www.pivotal.io/
650-215-0477
@GregChase
Blog: http://geekmarketing.biz/
Re: pdf of slides for Query and Index
Posted by Gregory Chase <gc...@pivotal.io>.
Attachment didn't come through. Please post on Slideshare.
Alternatively feel free to post editable form of the PPT to here:
https://github.com/Pivotal-Open-Source-Hub/POSH-Talks
-Greg
On Tue, Nov 3, 2015 at 11:21 AM, Jason Huynh <jh...@pivotal.io> wrote:
> Attached are the slides from todays talk.
>
> Please feel free to continue to post questions on the user or dev list and
> we will answer them as best as we can.
>
> As a follow up to one of the questions about cluster sizing and indexes
> that came up during todays talk:
> Although each case will depend on the data itself, here are some things to
> help keep in mind when calculating how much memory an index may take. This
> does not include any temporary objects/garbage created when executing a
> query or created when initially creating the index.
> Each index will also have additional overhead for managing the indexes per
> region but that size should be very small.
>
> Compact Functional Index:
> 1.) Reference for every region entry
> 2.) Size of extracted key
> 3.) Internal data structures (ConcurrentHashSet) used per index key
>
> Functional Index
> 1.) Size of copy of value and extracted key per entry to form the tuple
> 2.) Size of extracted key
> 3.) Internal data structures per index key, tuple structure
>
> Hash Index
> 1.) Reference per entry
> 2.) Internal array size
>
> Map Index
> 1.) Size of extracted key
> 2.) Size of internal map
> 3.) Sizes of Functional or Compact Index
>
> Primary Key
> 1.) Minor object creation
>
>
> Thanks,
> -Jason
>
>
>
>
--
Greg Chase
Director of Big Data Communities
http://www.pivotal.io/big-data
Pivotal Software
http://www.pivotal.io/
650-215-0477
@GregChase
Blog: http://geekmarketing.biz/