You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Emmanuel Lecharny <el...@apache.org> on 2008/06/30 16:15:26 UTC
Using derby BTree new implementation
Hi Kiran,
last spring we had an interesting convo with François Orsini about Derby
BTree implementation. As we have a new API on top of JDBM, and as they
have started to redisign their API to get it easier to use, it would be
interesting to look at what they are doing and see how we can leverage
this API. They are also interested to know what we think about the new
API, so any feedback would be good.
The best would for you two guys to get in touch, and see how you can
collaborate.
wdyt ?
--
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org
Re: Using derby BTree new implementation
Posted by Alex Karasulu <ak...@apache.org>.
Looks like they just need to see our Table and Index interfaces along with
the Cursor interface. We're essentially going to adapt their implementation
to these interfaces.
Alex
On Fri, Jul 4, 2008 at 2:50 AM, Kiran Ayyagari <ay...@gmail.com>
wrote:
> hi All,
>
> Francois sent a reply to my mail which am attaching here FYI. I yet to
> send a reply ( my be by tonight IST will
> send him)
>
> Please add any points if you have w.r.t Francois's mail.
>
> thanks,
> Kiran Ayyagari
>
>
> Emmanuel Lecharny wrote:
>
>> Alex Karasulu wrote:
>>
>>> Any releases out of this library yet? Got links?
>>>
>>>
>> All I have is
>> http://www.nabble.com/simpler-api-to-the-Derby-store-to18137499.html#a18137499
>>
>> I have been contacted by François Orsini directly, and transfered the mail
>> to Kiran and the dev list.
>>
>> I assume they want us to be guinea pigs, but with brain :) I like the
>> idea. It has many advantages :
>> - we get a direct line with the developpers
>> - we may get what we need in the API
>> - we have one more possible backend
>> - both project get a benefit out of it
>>
>> Bottom line is that we are not stuck with JDBM, but get a maintained BTree
>> code.
>>
>> I didn't had time to go any farther right now, and I think Kiran is the
>> guy when it comes to backend those days. I was just the french guy who
>> shared beers with another french fellow in Amsterdam, and discussed about
>> how our both projects can collaborate :) Wanna me to dig the thing inside
>> out ?
>>
>>
> --
> Kiran Ayyagari
>
> Hi Kiran,
>
> We're currently working on this and I hope we can provide you guys son
> enough with an implemented API for some early test drive.
>
> I have not posted in the directory dev mailing list as the API is not
> available yet.
>
> There is a thread in the Apache Derby dev forum as as far as the motivation
> for this and see how we can deliver it:
>
> http://www.nabble.com/simpler-api-to-the-Derby-store-to18137499.html#a18137499
>
> I'd be interested in knowing more about that new API you have on top of
> JDBM. Do you have any description and Javadoc for it? That would be great
> for us to see how it'd would work against the Derby B-Tree store one.
>
> Also may I ask what were/are the issues you are encountering with JDBM? I
> would assume that concurrency is likely one of them and also not being able
> to get new updates, etc.
>
> Thanks
>
> --francois
>
> Kiran Ayyagari wrote:
>
>>
>> hi Francois,
>>
>> I have searched directory dev mail archive but haven't found any
>> discussion threads in which you participated
>> ( infact there were some mails about BTree impl used in JDBM )
>>
>> Could you let me know about the derby revision in which this new API is
>> present ( will be glad if you can
>> provide some filenames related to BTree impl instead :) )
>>
>> Thanks for your time
>>
>> Kiran Ayyagari
>>
>> Emmanuel Lecharny wrote:
>>
>>> Hi Kiran,
>>>
>>> last spring we had an interesting convo with François Orsini about Derby
>>> BTree implementation. As we have a new API on top of JDBM, and as they have
>>> started to redisign their API to get it easier to use, it would be
>>> interesting to look at what they are doing and see how we can leverage this
>>> API. They are also interested to know what we think about the new API, so
>>> any feedback would be good.
>>>
>>> The best would for you two guys to get in touch, and see how you can
>>> collaborate.
>>>
>>> wdyt ?
>>>
>>>
>
>
>
Re: Using derby BTree new implementation
Posted by Kiran Ayyagari <ay...@gmail.com>.
hi All,
Francois sent a reply to my mail which am attaching here FYI. I yet to
send a reply ( my be by tonight IST will
send him)
Please add any points if you have w.r.t Francois's mail.
thanks,
Kiran Ayyagari
Emmanuel Lecharny wrote:
> Alex Karasulu wrote:
>> Any releases out of this library yet? Got links?
>>
> All I have is
> http://www.nabble.com/simpler-api-to-the-Derby-store-to18137499.html#a18137499
>
>
> I have been contacted by François Orsini directly, and transfered the
> mail to Kiran and the dev list.
>
> I assume they want us to be guinea pigs, but with brain :) I like the
> idea. It has many advantages :
> - we get a direct line with the developpers
> - we may get what we need in the API
> - we have one more possible backend
> - both project get a benefit out of it
>
> Bottom line is that we are not stuck with JDBM, but get a maintained
> BTree code.
>
> I didn't had time to go any farther right now, and I think Kiran is
> the guy when it comes to backend those days. I was just the french guy
> who shared beers with another french fellow in Amsterdam, and
> discussed about how our both projects can collaborate :) Wanna me to
> dig the thing inside out ?
>
--
Kiran Ayyagari
Re: Using derby BTree new implementation
Posted by Alex Karasulu <ak...@apache.org>.
On Fri, Jul 4, 2008 at 1:35 AM, Emmanuel Lecharny <el...@apache.org>
wrote:
> Alex Karasulu wrote:
>
>> Any releases out of this library yet? Got links?
>>
>>
> All I have is
> http://www.nabble.com/simpler-api-to-the-Derby-store-to18137499.html#a18137499
>
> I have been contacted by François Orsini directly, and transfered the mail
> to Kiran and the dev list.
>
> I assume they want us to be guinea pigs, but with brain :) I like the idea.
> It has many advantages :
> - we get a direct line with the developpers
> - we may get what we need in the API
> - we have one more possible backend
> - both project get a benefit out of it
>
All excellent points and we should support this effort. It's a no brainer
for us to implement and test for them. We can even get stats between a JDBM
and DerbyBTree partition implementation.
>
> Bottom line is that we are not stuck with JDBM, but get a maintained BTree
> code.
>
Aye options are great to have especially under the ASL.
>
> I didn't had time to go any farther right now, and I think Kiran is the guy
> when it comes to backend those days.
Aye he's doing great and has a full grasp of the partition design. I think
he could easily adapt it if he is interested and feels he can find the time.
> I was just the french guy who shared beers with another french fellow in
> Amsterdam, and discussed about how our both projects can collaborate :)
> Wanna me to dig the thing inside out ?
>
Just perfect let's keep this relationship progressing.
Thanks,
Alex
Re: Using derby BTree new implementation
Posted by Emmanuel Lecharny <el...@apache.org>.
Alex Karasulu wrote:
> Any releases out of this library yet? Got links?
>
All I have is
http://www.nabble.com/simpler-api-to-the-Derby-store-to18137499.html#a18137499
I have been contacted by François Orsini directly, and transfered the
mail to Kiran and the dev list.
I assume they want us to be guinea pigs, but with brain :) I like the
idea. It has many advantages :
- we get a direct line with the developpers
- we may get what we need in the API
- we have one more possible backend
- both project get a benefit out of it
Bottom line is that we are not stuck with JDBM, but get a maintained
BTree code.
I didn't had time to go any farther right now, and I think Kiran is the
guy when it comes to backend those days. I was just the french guy who
shared beers with another french fellow in Amsterdam, and discussed
about how our both projects can collaborate :) Wanna me to dig the thing
inside out ?
--
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org
Re: Using derby BTree new implementation
Posted by Alex Karasulu <ak...@apache.org>.
Any releases out of this library yet? Got links?
Alex
On Mon, Jun 30, 2008 at 10:15 AM, Emmanuel Lecharny <el...@apache.org>
wrote:
> Hi Kiran,
>
> last spring we had an interesting convo with François Orsini about Derby
> BTree implementation. As we have a new API on top of JDBM, and as they have
> started to redisign their API to get it easier to use, it would be
> interesting to look at what they are doing and see how we can leverage this
> API. They are also interested to know what we think about the new API, so
> any feedback would be good.
>
> The best would for you two guys to get in touch, and see how you can
> collaborate.
>
> wdyt ?
>
> --
> --
> cordialement, regards,
> Emmanuel Lécharny
> www.iktek.com
> directory.apache.org
>
>
>
Re: Using derby BTree new implementation
Posted by Kiran Ayyagari <ay...@gmail.com>.
hi Francois,
Let me first apologize for the late reply. more inline
Francois Orsini wrote:
> Hi Kiran,
>
> We're currently working on this and I hope we can provide you guys son
> enough with an implemented API for some early test drive.
>
> I have not posted in the directory dev mailing list as the API is not
> available yet.
>
> There is a thread in the Apache Derby dev forum as as far as the
> motivation for this and see how we can deliver it:
> http://www.nabble.com/simpler-api-to-the-Derby-store-to18137499.html#a18137499
>
>
> I'd be interested in knowing more about that new API you have on top
> of JDBM. Do you have any description and Javadoc for it? That would be
> great for us to see how it'd would work against the Derby B-Tree store
> one.
>
There are three top level interfaces namely Table, Index and Cursor.
A Table basically contains two columns <key, value> where keys are
sorted and both keys and values if
duplicate keys are allowed.
Index is a bidirectional lookup table ( internally maintains two
_Table_s)
A Cursor abstracts the JDBM specific browsing mechanism while also
adding support for traversal over
duplicate keys if they have been enabled in the Table.
FYI There is an excellent doc written by Alex explaining these
interfaces in detail which is present at
http://cwiki.apache.org/confluence/display/DIRxSRVx11/JDBM+Partition+Design
The SVN urls for
Table
http://svn.apache.org/viewvc/directory/apacheds/branches/bigbang/btree-base/src/main/java/org/apache/directory/server/xdbm/Table.java?revision=662411&view=markup
Index
http://svn.apache.org/viewvc/directory/apacheds/branches/bigbang/btree-base/src/main/java/org/apache/directory/server/xdbm/Index.java?revision=656489&view=markup
Cursor
http://svn.apache.org/viewvc/directory/apacheds/branches/bigbang/core-cursor/src/main/java/org/apache/directory/server/core/cursor/Cursor.java?revision=655158&view=markup
The corresponding implementation classes are
JdbmTable
http://svn.apache.org/viewvc/directory/apacheds/branches/bigbang/jdbm-store/src/main/java/org/apache/directory/server/core/partition/impl/btree/jdbm/JdbmTable.java?revision=662411&view=markup
JdbmIndex
http://svn.apache.org/viewvc/directory/apacheds/branches/bigbang/jdbm-store/src/main/java/org/apache/directory/server/core/partition/impl/btree/jdbm/JdbmIndex.java?revision=659719&view=markup
There are numerous implementations for the Cursor interface. The two
main cursor implementations are
NoDupsCursor
http://svn.apache.org/viewvc/directory/apacheds/branches/bigbang/jdbm-store/src/main/java/org/apache/directory/server/core/partition/impl/btree/jdbm/NoDupsCursor.java?revision=676922&view=markup
DupsCursor ( when duplicates are allowed in the Table )
http://svn.apache.org/viewvc/directory/apacheds/branches/bigbang/jdbm-store/src/main/java/org/apache/directory/server/core/partition/impl/btree/jdbm/DupsCursor.java?revision=656019&view=markup
> Also may I ask what were/are the issues you are encountering with
> JDBM? I would assume that concurrency is likely one of them and also
> not being able to get new updates, etc.
>
one issue ( that I know of ) with JDBM is its browser's getPrevious()
and getNext() when called in succession will
return the same key ( I was told that this not a bug but JDBM actually
works in this way )
e.x in a sequence of keys 1,2,3
assuming that browser is positioned at the beginning if we call
browser.getNext() (returns 1)
browser.getNext() (returns 2)
and finally browser.getPrevious() (returns *2* but logically speaking
should be 1)
ADS dev, please add any JDBM issues that you encountered or know about
> Thanks
>
> --francois
>
> Kiran Ayyagari wrote:
>>
>> hi Francois,
>>
>> I have searched directory dev mail archive but haven't found any
>> discussion threads in which you participated
>> ( infact there were some mails about BTree impl used in JDBM )
>>
>> Could you let me know about the derby revision in which this new
>> API is present ( will be glad if you can
>> provide some filenames related to BTree impl instead :) )
>>
>> Thanks for your time
>>
>> Kiran Ayyagari
>>
>> Emmanuel Lecharny wrote:
>>> Hi Kiran,
>>>
>>> last spring we had an interesting convo with François Orsini about
>>> Derby BTree implementation. As we have a new API on top of JDBM, and
>>> as they have started to redisign their API to get it easier to use,
>>> it would be interesting to look at what they are doing and see how
>>> we can leverage this API. They are also interested to know what we
>>> think about the new API, so any feedback would be good.
>>>
>>> The best would for you two guys to get in touch, and see how you can
>>> collaborate.
>>>
>>> wdyt ?
>>>
>
>
--
Kiran Ayyagari