You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Noah Slater <ns...@apache.org> on 2009/03/10 10:54:37 UTC
JSONQuery
Hey,
I thought some of you might find this interesting:
http://docs.persvr.org/documentation/jsonquery
XPath-like syntax for expressing selection queries against JSON data
structures. Interesting concept. I've always wondered why the basic concepts
behind XPath were never borrow and applied to other types of structured data —
it’s so insanely useful. I suppose jQuery popularized using CSS selectors for
querying HTML but why not take the same basic concept and apply it to problem
domains outside of SGML-inspired markup languages and their data models.
- Ryan Tomayko
Best,
--
Noah Slater, http://tumbolia.org/nslater
Re: JSONQuery
Posted by Dean Landolt <de...@deanlandolt.com>.
On Tue, Mar 10, 2009 at 7:46 AM, Robert Newson <ro...@gmail.com>wrote:
> fwiw: it might make a more natural query language into couchdb-lucene
> than Lucene's default query syntax. Not all of it applies, but there's
> some decent overlap.
>
> B.
>
> On Tue, Mar 10, 2009 at 12:40 PM, Dean Landolt <de...@deanlandolt.com>
> wrote:
> > On Tue, Mar 10, 2009 at 7:34 AM, Christopher Lenz <cm...@gmx.de> wrote:
> >
> >> On 10.03.2009, at 13:23, Dean Landolt wrote:
> >>
> >>> As an example of what I mean, JSONSchema already has an implementation
> in
> >>> another language (in this case, python again[1]). And JSONQuery even
> less
> >>> javascript-idiomatic. If you'd like, I'd be happy to try my hand at
> >>> implementing it in erlang -- but I was thinking it would be most
> helpful
> >>> for
> >>> view ops (thus, in js). It would be pretty convenient if there were
> >>> agreed-upon path semantics -- specifically for things like searching.
> >>>
> >>
> >> Everyone in this thread seems to know where/how JSONPath would fit into
> >> CouchDB, but I have no idea :)
> >>
> >> What would this be useful for?
> >
> >
> > One way -- retrieving doc fragments. Another -- for json-based
> configuration
> > of fti, rather than having to interpret js functions via rhino or
> > python-spidermonkey (no offense Paul -- that's still a kickass project
> ;).
> > There are probably quite a few others that will shake out -- but having
> an
> > agreed-upon way to reference portions of a doc (or a view) should prove
> > useful.
>
And one more use -- if and when alternate view engines land, many of them
will benefit from this. I know it's not a goal to make couch more *
relational*, but I can forsee some external indexing that will help solve a
lot of problems couch just can't solve (efficiently) on its own.
I've started experimenting with dropping couch on top of an xml db (monetdb
in this case). I know, I know -- xml...eww...but I work for an xml shop, so
what can i do? I'm finishing up a long project to build a restful data store
on top of MarkLogic -- it's a beautiful but obscenely expensive piece of
software, and I'd love to be able to kick them to the curb. Couch is a
natural fit out of the box, but there are a lot of relational operations we
have to support that couch can't (and shouldn't) do. With a little
alternative indexing we can fix this, and I think JSONQuery will help get me
there, so I'm a little biased :)
Re: JSONQuery
Posted by Robert Newson <ro...@gmail.com>.
fwiw: it might make a more natural query language into couchdb-lucene
than Lucene's default query syntax. Not all of it applies, but there's
some decent overlap.
B.
On Tue, Mar 10, 2009 at 12:40 PM, Dean Landolt <de...@deanlandolt.com> wrote:
> On Tue, Mar 10, 2009 at 7:34 AM, Christopher Lenz <cm...@gmx.de> wrote:
>
>> On 10.03.2009, at 13:23, Dean Landolt wrote:
>>
>>> As an example of what I mean, JSONSchema already has an implementation in
>>> another language (in this case, python again[1]). And JSONQuery even less
>>> javascript-idiomatic. If you'd like, I'd be happy to try my hand at
>>> implementing it in erlang -- but I was thinking it would be most helpful
>>> for
>>> view ops (thus, in js). It would be pretty convenient if there were
>>> agreed-upon path semantics -- specifically for things like searching.
>>>
>>
>> Everyone in this thread seems to know where/how JSONPath would fit into
>> CouchDB, but I have no idea :)
>>
>> What would this be useful for?
>
>
> One way -- retrieving doc fragments. Another -- for json-based configuration
> of fti, rather than having to interpret js functions via rhino or
> python-spidermonkey (no offense Paul -- that's still a kickass project ;).
> There are probably quite a few others that will shake out -- but having an
> agreed-upon way to reference portions of a doc (or a view) should prove
> useful.
>
Re: JSONQuery
Posted by Volker Mische <vo...@gmail.com>.
Dean Landolt wrote:
> On Tue, Mar 10, 2009 at 7:34 AM, Christopher Lenz <cm...@gmx.de> wrote:
>
>> On 10.03.2009, at 13:23, Dean Landolt wrote:
>>
>>> As an example of what I mean, JSONSchema already has an implementation in
>>> another language (in this case, python again[1]). And JSONQuery even less
>>> javascript-idiomatic. If you'd like, I'd be happy to try my hand at
>>> implementing it in erlang -- but I was thinking it would be most helpful
>>> for
>>> view ops (thus, in js). It would be pretty convenient if there were
>>> agreed-upon path semantics -- specifically for things like searching.
>>>
>> Everyone in this thread seems to know where/how JSONPath would fit into
>> CouchDB, but I have no idea :)
>>
>> What would this be useful for?
>
>
> One way -- retrieving doc fragments. Another -- for json-based configuration
> of fti, rather than having to interpret js functions via rhino or
> python-spidermonkey (no offense Paul -- that's still a kickass project ;).
> There are probably quite a few others that will shake out -- but having an
> agreed-upon way to reference portions of a doc (or a view) should prove
> useful.
>
Exactly, I use something similar (not as advanced as
XPath/JSONPath/JSONQuery) in GeoCouch for defining where in a document
the spatial information is stored [1]. I've called it pythonpath [2],
though I don't really like the name myself.
Cheers,
Volker
[1]
http://vmx.cx/cgi-bin/blog/index.cgi/geocouch-geospatial-queries-with-couchdb%3A2008-10-26%3Aen%2CCouchDB%2CPython%2Cgeo
[2]
http://vmx.cx/cgi-bin/blog/index.cgi/pythonpath-access-nested-data-structures:2008-11-08:en,Python
Re: JSONQuery
Posted by Paul Davis <pa...@gmail.com>.
On Tue, Mar 10, 2009 at 8:40 AM, Dean Landolt <de...@deanlandolt.com> wrote:
> On Tue, Mar 10, 2009 at 7:34 AM, Christopher Lenz <cm...@gmx.de> wrote:
>
>> On 10.03.2009, at 13:23, Dean Landolt wrote:
>>
>>> As an example of what I mean, JSONSchema already has an implementation in
>>> another language (in this case, python again[1]). And JSONQuery even less
>>> javascript-idiomatic. If you'd like, I'd be happy to try my hand at
>>> implementing it in erlang -- but I was thinking it would be most helpful
>>> for
>>> view ops (thus, in js). It would be pretty convenient if there were
>>> agreed-upon path semantics -- specifically for things like searching.
>>>
>>
>> Everyone in this thread seems to know where/how JSONPath would fit into
>> CouchDB, but I have no idea :)
>>
>> What would this be useful for?
>
Same as Dean, the one place I really saw using it a lot would be in
full text searching. I'm not *as* convinced for document fragments,
but if there were a somewhat standardized version I don't see any
reason to not support it.
>
> One way -- retrieving doc fragments. Another -- for json-based configuration
> of fti, rather than having to interpret js functions via rhino or
> python-spidermonkey (no offense Paul -- that's still a kickass project ;).
> There are probably quite a few others that will shake out -- but having an
> agreed-upon way to reference portions of a doc (or a view) should prove
> useful.
>
Heh. None taken and I agree whole heartedly.
Re: JSONQuery
Posted by Dean Landolt <de...@deanlandolt.com>.
On Tue, Mar 10, 2009 at 7:34 AM, Christopher Lenz <cm...@gmx.de> wrote:
> On 10.03.2009, at 13:23, Dean Landolt wrote:
>
>> As an example of what I mean, JSONSchema already has an implementation in
>> another language (in this case, python again[1]). And JSONQuery even less
>> javascript-idiomatic. If you'd like, I'd be happy to try my hand at
>> implementing it in erlang -- but I was thinking it would be most helpful
>> for
>> view ops (thus, in js). It would be pretty convenient if there were
>> agreed-upon path semantics -- specifically for things like searching.
>>
>
> Everyone in this thread seems to know where/how JSONPath would fit into
> CouchDB, but I have no idea :)
>
> What would this be useful for?
One way -- retrieving doc fragments. Another -- for json-based configuration
of fti, rather than having to interpret js functions via rhino or
python-spidermonkey (no offense Paul -- that's still a kickass project ;).
There are probably quite a few others that will shake out -- but having an
agreed-upon way to reference portions of a doc (or a view) should prove
useful.
Re: JSONQuery
Posted by Ben Browning <be...@gmail.com>.
Sorry to bring this thread back from the dead, but I just tried using
JSONQuery within a Couch view and everything seems to work fine. I'm
using the forked version of JSONQuery from here:
http://github.com/jcrosby/jsonquery/tree/master
The only gotcha is that JSONQuery expects either a namespace function
or a window object to exist. So, I just create a fake window object.
Here's an example of a working map view with JSONQuery.js included via
a CouchApp macro. You could just as easily copy/paste its contents
directly into a view if you aren't using CouchApp.
var window = {};
// !code JSONQuery.js
JSONQuery = window["JSONQuery"]
function(doc) {
var customers = JSONQuery("$.customers[?purchases > 21]", doc);
for (var i = 0; i < customers.length; i++) {
emit(customers[i].name, customers[i].purchases);
}
}
Re: JSONQuery
Posted by Chris Anderson <jc...@apache.org>.
On Tue, Mar 10, 2009 at 8:42 AM, kowsik <ko...@gmail.com> wrote:
> If i remember the original context, it's for pulling out document
> snippets using json-path's. So we don't have to write a view for each
> of the pieces of data we want.
>
> So if you have something like:
>
> {
> _id: "12345",
> comments: [ 'blah', 'foo' ],
> meta: {
> author: "me"
> }
> }
>
> GET /db/12345?path=meta/author
>
> will just return "me" instead of the entire document.
>
> K.
Which is a paradigm case for a reusable _show function. ;)
Chris
--
Chris Anderson
http://jchris.mfdz.com
Re: JSONQuery
Posted by kowsik <ko...@gmail.com>.
If i remember the original context, it's for pulling out document
snippets using json-path's. So we don't have to write a view for each
of the pieces of data we want.
So if you have something like:
{
_id: "12345",
comments: [ 'blah', 'foo' ],
meta: {
author: "me"
}
}
GET /db/12345?path=meta/author
will just return "me" instead of the entire document.
K.
On Tue, Mar 10, 2009 at 5:38 AM, Noah Slater <ns...@apache.org> wrote:
> On Tue, Mar 10, 2009 at 01:34:06PM +0100, Christopher Lenz wrote:
>> Everyone in this thread seems to know where/how JSONPath would fit into
>> CouchDB, but I have no idea :)
>
> Actually, I brought it up because most Couch Potatos use a lot of JSON.
>
> I don't really see how this would be used within CouchDB, but I may be wrong.
>
> --
> Noah Slater, http://tumbolia.org/nslater
>
Re: JSONQuery
Posted by Noah Slater <ns...@apache.org>.
On Tue, Mar 10, 2009 at 01:34:06PM +0100, Christopher Lenz wrote:
> Everyone in this thread seems to know where/how JSONPath would fit into
> CouchDB, but I have no idea :)
Actually, I brought it up because most Couch Potatos use a lot of JSON.
I don't really see how this would be used within CouchDB, but I may be wrong.
--
Noah Slater, http://tumbolia.org/nslater
Re: JSONQuery
Posted by Christopher Lenz <cm...@gmx.de>.
On 10.03.2009, at 13:23, Dean Landolt wrote:
> As an example of what I mean, JSONSchema already has an
> implementation in
> another language (in this case, python again[1]). And JSONQuery even
> less
> javascript-idiomatic. If you'd like, I'd be happy to try my hand at
> implementing it in erlang -- but I was thinking it would be most
> helpful for
> view ops (thus, in js). It would be pretty convenient if there were
> agreed-upon path semantics -- specifically for things like searching.
Everyone in this thread seems to know where/how JSONPath would fit
into CouchDB, but I have no idea :)
What would this be useful for?
Thanks,
--
Christopher Lenz
cmlenz at gmx.de
http://www.cmlenz.net/
Re: JSONQuery
Posted by Dean Landolt <de...@deanlandolt.com>.
On Tue, Mar 10, 2009 at 7:12 AM, Jan Lehnardt <ja...@apache.org> wrote:
>
> On 10 Mar 2009, at 13:07, Paul Davis wrote:
>
> On Tue, Mar 10, 2009 at 7:59 AM, Dean Landolt <de...@deanlandolt.com>
>> wrote:
>>
>>> On Tue, Mar 10, 2009 at 6:50 AM, Noah Slater <ns...@apache.org> wrote:
>>>
>>> On Tue, Mar 10, 2009 at 07:43:16AM -0400, Paul Davis wrote:
>>>>
>>>>> Haven't we discussed this before? Or am I just getting a weird déjà vu?
>>>>>
>>>>
>>>> Well, we've certainly discussed JSON paths, JSON queries, and JSON
>>>> diffs.
>>>>
>>>> I didn't think this specific suggestion has come up before, apologies.
>>>>
>>>> I still think this is low hanging RFC fruit. Heh
>>>>
>>>
>>>
>>> We had discussed JSONPath, which at first glance this is, but it looks
>>> like
>>> they cleaned it up quite a bit. It's even got pythonic list slices so
>>> this
>>> is definitely *not* javascript -- it's just that the reference
>>> implementation is in js, and that's the only implementation...for now.
>>>
>>> Looking at the new spec, I've already implemented half of this in python
>>> --
>>> I'd be happy to take it all the way if this would help formalize path
>>> semantics in couch.
>>>
>>>
>> If its done in JS its probably done in Python [1] (shameless plug).
>> The bigger issue is how to get an Erlang implementation. I mentioned
>> to Jan about trying to link spidermonkey directly into the Erlang VM.
>> He said some guys have tried and it ended in abandonment because JS
>> was too unstable. Though it does sound like a fun experiment.
>>
>
> "Some Guys" being the original Erlang & OTP developers :) They do not
> recommend going down that route.
What do they know ;)
But actually Paul, I was just mentioning a pure python parser as a proof of
concept to show JSONQuery doesn't need js. In fact, if you look at the new
spec Noah linked to, it's not even *close *to js (it strikes me as more like
a jumble of python and ruby with some xpath tossed in).
As an example of what I mean, JSONSchema already has an implementation in
another language (in this case, python again[1]). And JSONQuery even less
javascript-idiomatic. If you'd like, I'd be happy to try my hand at
implementing it in erlang -- but I was thinking it would be most helpful for
view ops (thus, in js). It would be pretty convenient if there were
agreed-upon path semantics -- specifically for things like searching.
[1] http://code.google.com/p/jsonschema/
Re: JSONQuery
Posted by Jan Lehnardt <ja...@apache.org>.
On 10 Mar 2009, at 13:07, Paul Davis wrote:
> On Tue, Mar 10, 2009 at 7:59 AM, Dean Landolt <de...@deanlandolt.com>
> wrote:
>> On Tue, Mar 10, 2009 at 6:50 AM, Noah Slater <ns...@apache.org>
>> wrote:
>>
>>> On Tue, Mar 10, 2009 at 07:43:16AM -0400, Paul Davis wrote:
>>>> Haven't we discussed this before? Or am I just getting a weird
>>>> déjà vu?
>>>
>>> Well, we've certainly discussed JSON paths, JSON queries, and JSON
>>> diffs.
>>>
>>> I didn't think this specific suggestion has come up before,
>>> apologies.
>>>
>>> I still think this is low hanging RFC fruit. Heh
>>
>>
>> We had discussed JSONPath, which at first glance this is, but it
>> looks like
>> they cleaned it up quite a bit. It's even got pythonic list slices
>> so this
>> is definitely *not* javascript -- it's just that the reference
>> implementation is in js, and that's the only implementation...for
>> now.
>>
>> Looking at the new spec, I've already implemented half of this in
>> python --
>> I'd be happy to take it all the way if this would help formalize path
>> semantics in couch.
>>
>
> If its done in JS its probably done in Python [1] (shameless plug).
> The bigger issue is how to get an Erlang implementation. I mentioned
> to Jan about trying to link spidermonkey directly into the Erlang VM.
> He said some guys have tried and it ended in abandonment because JS
> was too unstable. Though it does sound like a fun experiment.
"Some Guys" being the original Erlang & OTP developers :) They do not
recommend going down that route.
Cheers
Jan
-
Re: JSONQuery
Posted by Paul Davis <pa...@gmail.com>.
On Tue, Mar 10, 2009 at 7:59 AM, Dean Landolt <de...@deanlandolt.com> wrote:
> On Tue, Mar 10, 2009 at 6:50 AM, Noah Slater <ns...@apache.org> wrote:
>
>> On Tue, Mar 10, 2009 at 07:43:16AM -0400, Paul Davis wrote:
>> > Haven't we discussed this before? Or am I just getting a weird déjà vu?
>>
>> Well, we've certainly discussed JSON paths, JSON queries, and JSON diffs.
>>
>> I didn't think this specific suggestion has come up before, apologies.
>>
>> I still think this is low hanging RFC fruit. Heh
>
>
> We had discussed JSONPath, which at first glance this is, but it looks like
> they cleaned it up quite a bit. It's even got pythonic list slices so this
> is definitely *not* javascript -- it's just that the reference
> implementation is in js, and that's the only implementation...for now.
>
> Looking at the new spec, I've already implemented half of this in python --
> I'd be happy to take it all the way if this would help formalize path
> semantics in couch.
>
If its done in JS its probably done in Python [1] (shameless plug).
The bigger issue is how to get an Erlang implementation. I mentioned
to Jan about trying to link spidermonkey directly into the Erlang VM.
He said some guys have tried and it ended in abandonment because JS
was too unstable. Though it does sound like a fun experiment.
HTH,
Paul Davis
[1] http://github.com/davisp/python-spidermonkey
Re: JSONQuery
Posted by Dean Landolt <de...@deanlandolt.com>.
On Tue, Mar 10, 2009 at 6:50 AM, Noah Slater <ns...@apache.org> wrote:
> On Tue, Mar 10, 2009 at 07:43:16AM -0400, Paul Davis wrote:
> > Haven't we discussed this before? Or am I just getting a weird déjà vu?
>
> Well, we've certainly discussed JSON paths, JSON queries, and JSON diffs.
>
> I didn't think this specific suggestion has come up before, apologies.
>
> I still think this is low hanging RFC fruit. Heh
We had discussed JSONPath, which at first glance this is, but it looks like
they cleaned it up quite a bit. It's even got pythonic list slices so this
is definitely *not* javascript -- it's just that the reference
implementation is in js, and that's the only implementation...for now.
Looking at the new spec, I've already implemented half of this in python --
I'd be happy to take it all the way if this would help formalize path
semantics in couch.
Re: JSONQuery
Posted by Noah Slater <ns...@apache.org>.
On Tue, Mar 10, 2009 at 07:43:16AM -0400, Paul Davis wrote:
> Haven't we discussed this before? Or am I just getting a weird déjà vu?
Well, we've certainly discussed JSON paths, JSON queries, and JSON diffs.
I didn't think this specific suggestion has come up before, apologies.
I still think this is low hanging RFC fruit. Heh
--
Noah Slater, http://tumbolia.org/nslater
Re: JSONQuery
Posted by Dean Landolt <de...@deanlandolt.com>.
On Tue, Mar 10, 2009 at 6:43 AM, Paul Davis <pa...@gmail.com>wrote:
> Haven't we discussed this before? Or am I just getting a weird déjà vu?
>
> Anyway, the way I remember it going was that its neat but requires a
> full JavaScript VM to meet the spec.
Yes, and that was the general consensus, but it may be worth reconsidering.
Sure, in order to have a already-working JSONPath (or JSONQuery -- whatever
Kris is calling it these days) implementation you need a js vm, but when I
learned that there's a nearly-complete jquery implementation in python [1],
it occured to me it couldn't be that hard to implement all (or a useful part
of) the JSONPath spec in any JSON-aware (read: any) language.
[1] http://pyquery.org/
Re: JSONQuery
Posted by Paul Davis <pa...@gmail.com>.
Haven't we discussed this before? Or am I just getting a weird déjà vu?
Anyway, the way I remember it going was that its neat but requires a
full JavaScript VM to meet the spec.
HTH,
Paul Davis
On Tue, Mar 10, 2009 at 5:54 AM, Noah Slater <ns...@apache.org> wrote:
> Hey,
>
> I thought some of you might find this interesting:
>
> http://docs.persvr.org/documentation/jsonquery
>
> XPath-like syntax for expressing selection queries against JSON data
> structures. Interesting concept. I've always wondered why the basic concepts
> behind XPath were never borrow and applied to other types of structured data —
> it’s so insanely useful. I suppose jQuery popularized using CSS selectors for
> querying HTML but why not take the same basic concept and apply it to problem
> domains outside of SGML-inspired markup languages and their data models.
>
> - Ryan Tomayko
>
> Best,
>
> --
> Noah Slater, http://tumbolia.org/nslater
>