Filtered multi-column lists

[Kore Nordmann <ma...@kore-nordmann.de>]

Hi list,

I love CouchDB and it is really fun using it, but now I am experience a 
problem which seems not really solveable properly to me at a first 
glance using the CouchDB views.

I have some amount of documents, which do have several fields in common, 
which each has some small amount of different values, just like an issue 
tracker, where you have for example bug type, status and priority for 
your documents.

The amount of documents may be "big" at some point (something between 
10^3 and 10^6), so I don't see ad hoc views as a good solution, as the 
results probably would take too much time to calculate. Also for the 
amount of fields and their different values it seems not feasible to 
define a view for each possible combination.

I also do not see a real solution in complex keys, like array keys with 
the provided view values, like [$type, $status, $priority], because 
afaik it is yet impossible to do queries like [*, (urgent | low), *] or 
similar.

Am I overlooking something here, or isn't there a good solution for such 
a use case yet?

Even I can't find the discussion on the ML now, I remember discussion on 
XPath-like queries for JSON structures, which could help here for 
querying the keys of a view, when it holds all the relevant information 
in its keys - even it may take quite some time to perform such a query 
against all keys in a view.

If there isn't something like this planned yet, and I am not overlooking 
some kind of obvious solution - do you have some good starting points 
which could offer a solid base for some evaluation work?

-- 
Kore Nordmann                                        (GPG 0xDDC70BBB)
http://kore-nordmann.de/portfolio.html

Re: Filtered multi-column lists

[Chris Anderson <jc...@grabb.it>]

On Wed, Jun 25, 2008 at 5:19 AM, Kore Nordmann <ma...@kore-nordmann.de> wrote:

>
> I also do not see a real solution in complex keys, like array keys with the
> provided view values, like [$type, $status, $priority], because afaik it is
> yet impossible to do queries like [*, (urgent | low), *] or similar.

The best solution for you may be to have individual views for each
queryable field: one each for sorting on $type, $status, and $priority
- and you may want these views to have complex keys if it is common to
query for eg $component, $status or perhaps $assignee, $priority

There is no real shortcut here - you'll need a view that puts records
in contiguous order for the major queries you'll be running. Or you
can start working on alternate indexing scheme's (something like the
wildcard syntax you mentioned sure would be handy, and probably "fast
enough" on smaller data sets.)


-- 
Chris Anderson
http://jchris.mfdz.com

Re: Filtered multi-column lists

[Paul Bonser <mi...@gmail.com>]

On Wed, Jun 25, 2008 at 7:19 AM, Kore Nordmann <ma...@kore-nordmann.de> wrote:
> I also do not see a real solution in complex keys, like array keys with the
> provided view values, like [$type, $status, $priority], because afaik it is
> yet impossible to do queries like [*, (urgent | low), *] or similar.

You could probably get about halfway there using startkey and endkey,
though this would only work for contiguous sets (low and medium,
medium and high, etc..).

It would probably be nice to add the ability to request a specific
list of keys. I wonder if that would be any less efficient than
specifying a single key.

I'm pretty new to CouchDB, but I don't think that there is a solution
yet, other than creating a separate view for each possible
permutation, which works for a few possible values, but would take a
lot of space for indexes for anything too big.


-- 
Paul Bonser
http://blog.paulbonser.com