[GitHub] [couchdb] davisp commented on issue #2456: FDB Builtin reduce

[GitBox <gi...@apache.org>]

davisp commented on issue #2456: FDB Builtin reduce
URL: https://github.com/apache/couchdb/pull/2456#issuecomment-579458016
 
 
   I finally had some downtime to start looking into this more closely. So far I've only spent a couple hours reading through and thinking about things so hopefully this comes out semi-coherently.
   
   So far I'm just focusing on the skiplist implementation. First off, as @chewbranca's confusion points out, we should probably scrub all references to skiplists because these are different enough that it'll just lead to confusion when people attempt to compare them. I'd at least mention RankedSets and include links to the Java implementation in fdb-record-layer [1]. RankedSets is a bit overly specific given we've got arbitrary reduce functions. But maybe `reduced_sets` would be good enough?
   
   I started reading on the update side of things first and it looks like you've basically nailed it. I think we should change around a few definitions though. If we refer to the map index's key/value pairs as level 0 and then keep the rest as written we'll have covered custom javascript reduce functions and all of the builtin reduce functions. Which is way more awesome than when I was thinking we were going to have to invent some sort of un-reduce function which would have been super error prone. Brilliant work there!
   
   The query/traversal side of things is going to need some work. I think there's a bit of confusion around how group levels work that got mixed into the traversal code. To simplify the implementation and development here you should write a function that will correctly answer a group=true query with a start and end key. Once you have an implementation of that function, turning it into a `foldl` style API will be straightforward. And then once you have that API you'll basically just be copying the group logic from couch_btree. The algorithm that @chewbranca sketched should be a good starting place here.
   
   Once you've got all that done, adding the removal logic is as simple as cribbing off the Java `RankedSet.remove` method.
   
   That's about it from me for now. Good work!
   
   [1] https://github.com/FoundationDB/fdb-record-layer/blob/master/fdb-extensions/src/main/java/com/apple/foundationdb/async/RankedSet.java

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services