Declarative validation of recursive documents

[Daniel Itaboraí <it...@gmail.com>]

I'm pretty new to Couch and I'm trying to let go of my relational footing
for a little bit. This is going to sound a little like a crackpot rant, so
please bear with me for a second.

Relational databases are all about meaningfully linking simple tuples. The
validity of the "data graph" is maintained basically by how you identify and
how you relate such tuples.

In a document database all document are tree like structures. The database
itself is like a forrest of such trees. There is no joining. The validity of
a document is all in its structure and the validity of the "data forrest" is
achieved by making sure that every document adheres to some sort of tree
like schema.

I'm starting to think that "the fatter, the better" for documents, as along
as you don't hit some operational hiccups such as high probability of
concurrent writes.

It seems to me that it would be useful to be able to declaratively specify
this tree like schema to validate complex documents, something like the
following

    Doc(
        Field('left', Recursive()),
        Field('value', Number()),
        Field('right', Recursive())
    ).validate({
        'value': 10,
        'left': {
            'value': 5,
            'left': {'value': 2},
            'right': {'value': 6, 'right': {'value': 7}}
        }
    })

    or

    Doc(
        Field('subject', String()),
        Field('body', String()),
        Field('replies', Array(Recursive()))
    ).validate({
        'subject': 'Couch rulez',
        'body': '... and yet I suck',
        'replies': [
            {
                'subject': 'Re: Couch rulez',
                'body': 'this!',
                'replies': [{'subject': 'Re: Re: Couch rulez', 'body':
'Totally', 'replies':[]}]
            },
            {
                'subject': 'Spam',
                'body': 'Buy something crappy',
                'replies': []
            }
        ]
    })

I banged away at the keyboard until I got this
http://pastebin.com/N8cv1mYn(it ain't pretty and is probably filled
with errors) ... and I'm not sure if
it is something worth pursuing.

Is this sort of thing useful? And more importantly, is there anything
that already does this better?

Thanks,
Daniel

Re: Declarative validation of recursive documents

[Roger Rohrbach <ro...@ecstatic.com>]

See also Orderly, a really nice, terse syntax that compiles into JSON Schema.  I've used it to specify the documents for a project using CouchDB.  The bad news is: the developer seems to have abandoned the project, while JSON Schema continues to evolve.  My attempts at engaging the developer have been met with silence.  The good news is: anyone can fork it on github and take it further.

Re: Declarative validation of recursive documents

[Jens Alfke <je...@couchbase.com>]

On Sep 12, 2011, at 7:37 PM, Daniel Itaboraí wrote:

It seems to me that it would be useful to be able to declaratively specify
this tree like schema to validate complex documents, something like the
following

   Doc(
       Field('left', Recursive()),
       Field('value', Number()),

It looks as though you’re on the way to re-inventing JSON Schemas <http://json-schema.org/> :) I just ran into that last weekend and was thinking it’d be useful to use with CouchDB.

—Jens

Re: Declarative validation of recursive documents

[Mark Hahn <ma...@boutiquing.com>]

> Is this sort of thing useful?

I don't think I would find it useful because of my couch coding philosophy.
 I write "hardened" code such that any document can have any garbage and the
code does the best it can and fails gracefully.  I allow partial documents
that are "unfinished".  I find this approach to be a refreshing change from
structured DBs and kind of matches the nosql philosophy.  So ensuring
correctness at the doc level would not be a match for my usage.

I do realize that many coders would be horrified by my approach.

Re: Declarative validation of recursive documents

[Randall Leeds <ra...@gmail.com>]

2011/9/12 Daniel Itaboraí <it...@gmail.com>

> It seems to me that it would be useful to be able to declaratively specify
> this tree like schema to validate complex documents, something like the
> following
>
> <snip>

>
> Is this sort of thing useful? And more importantly, is there anything
> that already does this better?
>

CouchDB-python[1] has a mapping module[2].
Mikeal's couchdb-pythonviews[3] is a python view server that supports
validation functions.

Depending on your needs, you might accomplish the job with just these.

For example, if you're okay with accepting documents where all the fields
coerce to their expected types even if they were not exact matches, you
could just use the couchdb.mapping.Document.wrap on input to your validation
function. This would, for example, generate exceptions when a DateField
fails to parse as a date.

If you need something more strict, like rejecting JSON numbers where you
expect strings, you could probably add python type information to the
mapping classes and a validate() function to the base class.

Hope that's enough to get you started. I'm not aware of anything like this
that exists already, but if it does hopefully the list will chime in.

[1] https://code.google.com/p/couchdb-python/
[2]
https://code.google.com/p/couchdb-python/source/browse/couchdb/mapping.py
[3] https://github.com/mikeal/couchdb-pythonviews


> Thanks,
> Daniel
>