Gathering data through recursive references

[Nils Breunese <N....@vpro.nl>]

Hallo all,

I have written a couple of CouchDB map functions by now, which are being 
used in production. All is fine there. The documents in my database are 
JSON-serialized objects from a Java application. And I think I have come 
to a point where this setup is breaking down.

I'll try to simplify my problem as much as possible. I have two types of 
documents: group documents with type:"group" and program documents with 
type: "program". Programs can be episodes of groups, and groups can be 
members of other groups.

A bare-bones program document might look like this:

----
{
     "_id": "program:101",
     "_rev": "3-4139e2a50b76a736b676475f960596c0",
     "episodeOf": [
         {
             "position": 26,
             "reference": "group:3"
         }
     ],
     "type": "program"
}
----

The referenced group document might look like this:

----
{
     "_id": "group:3",
     "_rev": "1-0e7eff25b91e1e5d93ca19655c2afa69",
     "memberOf": [
         {
             "position": 1,
             "reference": "group:1"
         }
     ],
     "type": "group"
}
----

And the group document referenced in the above group document might be:

----
{
     "_id": "group:1",
     "_rev": "1-1899b6390756e2f121efbd85a04ea137",
     "type": "group"
}
----

Now I need to gather all the program data for a group (say, all programs 
that are an episode of 'group:3' or 'group:1'). Does this require 
chained map/reduce (which is not in CouchDB yet, but seems to come up 
more and more on this mailinglist) or is there another way to do this?

Nils.

Re: Gathering data through recursive references

[ko...@fillibach.de]

Quoting Nils Breunese <N....@vpro.nl>:
> Kosta wrote:
>> Quoting Nils Breunese <N....@vpro.nl>:

>> * Recursive Queries (i.e. ask for all programs in group x and use   
>> recursive queries to check whether x is actually in the path z-y-x)
>> * Storing the whole path in the document (which is a pain in case  
>> you  ever move the group)
> Thanks for confirming my thoughts. I'll see if I can get the  
> incoming data to change so the whole path is stored. Otherwise the  
> clients will indeed need to use multiple queries.

I thought about this a little more. Either the group name itself is a  
unique identifier, then you only need one query per group (there's no  
group x OR group y), but you need recursive queries to construct the  
path. Or group names can be ambiguous (but the path is unique), then  
you need the full path stored in each doc anyway.

HTH,
Kosta

Re: Gathering data through recursive references

[Nils Breunese <N....@vpro.nl>]

Kosta wrote:

> Quoting Nils Breunese <N....@vpro.nl>:
> 
>> Now I need to gather all the program data for a group (say, all  
>> programs that are an episode of 'group:3' or 'group:1'). Does this  
>> require chained map/reduce (which is not in CouchDB yet, but seems  
>> to come up more and more on this mailinglist) or is there another  
>> way to do this?
> 
> The way to do this either are
> * Recursive Queries (i.e. ask for all programs in group x and use  
> recursive queries to check whether x is actually in the path z-y-x)
> * Storing the whole path in the document (which is a pain in case you  
> ever move the group)

Thanks for confirming my thoughts. I'll see if I can get the incoming 
data to change so the whole path is stored. Otherwise the clients will 
indeed need to use multiple queries.

Nils.

Re: Gathering data through recursive references

[ko...@fillibach.de]

Quoting Nils Breunese <N....@vpro.nl>:

> Now I need to gather all the program data for a group (say, all  
> programs that are an episode of 'group:3' or 'group:1'). Does this  
> require chained map/reduce (which is not in CouchDB yet, but seems  
> to come up more and more on this mailinglist) or is there another  
> way to do this?

The way to do this either are
* Recursive Queries (i.e. ask for all programs in group x and use  
recursive queries to check whether x is actually in the path z-y-x)
* Storing the whole path in the document (which is a pain in case you  
ever move the group)

HTH,
Kosta