Merge heads-up

[Emmanuel Lécharny <el...@gmail.com>]

Hi guys,

sorry for having been silent those last 10 days, I had many private 
things to manage, plus I wantd to take a bit of a break to calmy think 
about our options.

I had a long convo with Alex last week (or the week before, I don't 
remember), about the problem we had with the merge, and the various 
issues we had with the txn layer (at least, the pb I was perceiving when 
I had this convo). Bottom line, I think I freaked out about the 
coplexity of th emerge, plus the fact that some serious point needed to 
be handled if we switch to the txn layer.

Here is what I think as of today :
- there is no mean to merge the branch into the trunk : it will be way 
easier to do the opposite, simply because I *know* what I changed in the 
trunk, also because I have already merged the trunk into the branch many 
times, so it will be easier to do so
- it will take a bit of time, but that worth the effort
- in any case, the txn-layer is the oly exit strategy we have right now. 
I have tried to think about a better solution that would help us in a 
very short delay, but there is none.
- the problem we have in the txn layer can be mitigated with time, and 
we have plenty, as soon as we can deliver a server that does not have 
concurrent access issues

Now, we do have some concerns to take care of in the near future, if the 
txn branch becomes the new trunk :
- the fact that we handle MVCC in memory is problematic, as we may keep 
in memory a hell lots of page : a single modification will copy log(N) 
pages in memory, if N is the number of elements we have in the btree. As 
we update many indexes (7 at least), that mean we will keep 7xLog(N) 
pages, each page being at least 4Ko (and this can be more for entries.
- if someone does not close a cursor, or even worse, if an external 
client does not close a NamingEnumeration, then we will quickly get an 
OOM on the server, or the cache will just block, forbidding any 
modification to occur.

I'm really concerned by the second point.

Anyway, as a side effort, I have started to work on some B+Tree MVCC 
project (mavibot labs), because I wanted to play around some new ideas, 
after having read many papers about some new algorithms 
(http://xxx.lanl.gov/abs/1103.4282) that may be very interesting to 
implement. It's also my perception that such a project would be very 
useful for many other projects than just Directory. Anwyay, this is 
refreshing for me.

That's pretty much, I wanted to get you informed about what I was doing, 
but keep in mind that I'm also 80% busy on side tasks. Sadly, I'm not 
sure I can be full speed on ADS for at least one month or a bit more.

Thanks !

-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com