You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@asterixdb.apache.org by Ian Maxon <im...@uci.edu> on 2016/11/29 23:17:12 UTC
Dev notes about storage file/folder layouts
Hi all,
I went ahead and made a quick document describing how all the storage files
get laid out, since right now I don't think anything like that exists. It's
something we should have described at least a little, as otherwise the only
way to know is to read the code. It is good to know what to expect in those
folders for diagnosing lots of storage/persistence related issues.
The document is here:
https://cwiki.apache.org/confluence/display/ASTERIXDB/AsterixDB+On-Disk+File+and+Folder+structure
.
Suggestions/corrections are more than welcome of course.
- Ian
Re: Dev notes about storage file/folder layouts
Posted by Till Westmann <ti...@apache.org>.
There\u2019s a change that would change one aspect of this:
https://asterix-gerrit.ics.uci.edu/#/c/1367/
Thoughts welcome.
Cheers,
Till
On 29 Nov 2016, at 15:53, Ian Maxon wrote:
> A good thought, much appreciated. I've added the two missing suffixes to
> the document.
>
> On Tue, Nov 29, 2016 at 3:37 PM, Taewoo Kim <wa...@gmail.com> wrote:
>
>> It looks good! One comment: it would be nice to add all possible suffixes
>> for each index type.
>>
>> Common
>> BLOOM_FILTER_STRING = f
>>
>> B-Tree
>> BTREE_STRING = b
>> BUDDY_BTREE_STRING = buddy
>>
>> Inverted Index
>> DICT_BTREE_SUFFIX = b
>> INVLISTS_SUFFIX = i
>> DELETED_KEYS_BTREE_SUFFIX = d
>>
>> R-Tree
>> RTREE_STRING = r
>> BTREE_STRING = b
>>
>>
>>
>> Best,
>> Taewoo
>>
>> On Tue, Nov 29, 2016 at 3:17 PM, Ian Maxon <im...@uci.edu> wrote:
>>
>>> Hi all,
>>> I went ahead and made a quick document describing how all the storage
>> files
>>> get laid out, since right now I don't think anything like that exists.
>> It's
>>> something we should have described at least a little, as otherwise the
>> only
>>> way to know is to read the code. It is good to know what to expect in
>> those
>>> folders for diagnosing lots of storage/persistence related issues.
>>>
>>> The document is here:
>>> https://cwiki.apache.org/confluence/display/ASTERIXDB/
>>> AsterixDB+On-Disk+File+and+Folder+structure
>>> .
>>>
>>> Suggestions/corrections are more than welcome of course.
>>>
>>> - Ian
>>>
>>
Re: Dev notes about storage file/folder layouts
Posted by Ian Maxon <im...@uci.edu>.
A good thought, much appreciated. I've added the two missing suffixes to
the document.
On Tue, Nov 29, 2016 at 3:37 PM, Taewoo Kim <wa...@gmail.com> wrote:
> It looks good! One comment: it would be nice to add all possible suffixes
> for each index type.
>
> Common
> BLOOM_FILTER_STRING = f
>
> B-Tree
> BTREE_STRING = b
> BUDDY_BTREE_STRING = buddy
>
> Inverted Index
> DICT_BTREE_SUFFIX = b
> INVLISTS_SUFFIX = i
> DELETED_KEYS_BTREE_SUFFIX = d
>
> R-Tree
> RTREE_STRING = r
> BTREE_STRING = b
>
>
>
> Best,
> Taewoo
>
> On Tue, Nov 29, 2016 at 3:17 PM, Ian Maxon <im...@uci.edu> wrote:
>
> > Hi all,
> > I went ahead and made a quick document describing how all the storage
> files
> > get laid out, since right now I don't think anything like that exists.
> It's
> > something we should have described at least a little, as otherwise the
> only
> > way to know is to read the code. It is good to know what to expect in
> those
> > folders for diagnosing lots of storage/persistence related issues.
> >
> > The document is here:
> > https://cwiki.apache.org/confluence/display/ASTERIXDB/
> > AsterixDB+On-Disk+File+and+Folder+structure
> > .
> >
> > Suggestions/corrections are more than welcome of course.
> >
> > - Ian
> >
>
Re: Dev notes about storage file/folder layouts
Posted by Taewoo Kim <wa...@gmail.com>.
It looks good! One comment: it would be nice to add all possible suffixes
for each index type.
Common
BLOOM_FILTER_STRING = f
B-Tree
BTREE_STRING = b
BUDDY_BTREE_STRING = buddy
Inverted Index
DICT_BTREE_SUFFIX = b
INVLISTS_SUFFIX = i
DELETED_KEYS_BTREE_SUFFIX = d
R-Tree
RTREE_STRING = r
BTREE_STRING = b
Best,
Taewoo
On Tue, Nov 29, 2016 at 3:17 PM, Ian Maxon <im...@uci.edu> wrote:
> Hi all,
> I went ahead and made a quick document describing how all the storage files
> get laid out, since right now I don't think anything like that exists. It's
> something we should have described at least a little, as otherwise the only
> way to know is to read the code. It is good to know what to expect in those
> folders for diagnosing lots of storage/persistence related issues.
>
> The document is here:
> https://cwiki.apache.org/confluence/display/ASTERIXDB/
> AsterixDB+On-Disk+File+and+Folder+structure
> .
>
> Suggestions/corrections are more than welcome of course.
>
> - Ian
>