You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Chris Anderson <jc...@apache.org> on 2009/01/19 08:55:31 UTC
all_seqs stream (was Re: Update notifications including update sequence)
There are some other update-notification features that could be really
useful, especially for streaming applications.
Damien's called for a Comet-style connection option for replication,
which brings really cool affordances on the browser (as well as
industrial strength replication-streams).
Essentially we're talking about an all_seqs view, with various models
for consuming it.
/db/_all_seqs?since=200
This would send you the updates that happen after seq 200. With this
you can hold open a connection, waiting for sequence numbers to
increment.
You can filter the updates
There is also the possibility of filtering the updates with a
JavaScript (or other) function posted to the server. It returns true
or false, passing matching updates along to the client. Updates that
don't match won't be sent to the client. Using this method you can get
instant updates of eg: my emails.
This is all Damien's magic, basically things he's mentioned on IRC
etc. I think Couch is ready for us to start refining the client APIs.
There's room to do some fairly heavy lifting here.
--
Chris Anderson
http://jchris.mfdz.com
Re: all_seqs stream (was Re: Update notifications including update sequence)
Posted by Ulises <ul...@gmail.com>.
> The _external would still need to be able to respond to client
> requests - although processes (like an index updater) that don't
> directly talk to the client could work just fine without being
> registered as a db-update-notification-process.
That's what I meant, sorry for not being clear. I was probably
thinking more about things like FTI which do an extra layer of
processing on the documents and not much interaction with users
(unless queried).
> There is also some value to having Couch agree to kick a set of
> scripts when updates come in (in case something dies), but if the
> process management code starts to bog us down we can always drop it
> and refer people to their operating system's services support.
Agree. Still I see a real benefit in being able to run an indexer on a
totally different host than the one couchdb is running on.
> I got excited to write this thread when I started drafting out what
> IRC would look like if it ran on Couches. :)
it would look like teh wins?
U
Re: all_seqs stream (was Re: Update notifications including update sequence)
Posted by Chris Anderson <jc...@apache.org>.
On Mon, Jan 19, 2009 at 12:35 AM, Ulises <ul...@gmail.com> wrote:
>> The idea behind this is to unify the replication and standard http
>> client interfaces for updates streams, so for instance GET replication
>> (incrementally updating a db that's current as of seq 200) could work
>> via a request like:
>>
>> /db/_updates?since=200&include_docs=full
>>
>> And an _external that wanted to see every (or a subset) of updates
>> (for instance a Full Text Indexer) could use the same mechanism to
>> incrementally maintain an index.
>
> That's rather interesting and beg the question whether this wouldn't
> render _external obsolete? You could simply manage your own
> ex-_external as a separate process that kept this comet notification
> channel open.
The _external would still need to be able to respond to client
requests - although processes (like an index updater) that don't
directly talk to the client could work just fine without being
registered as a db-update-notification-process.
There is also some value to having Couch agree to kick a set of
scripts when updates come in (in case something dies), but if the
process management code starts to bog us down we can always drop it
and refer people to their operating system's services support.
>
>> Update streams (especially filtered ones) are one of the most powerful
>> tools CouchDB offers. For instance, with filtered streams, you can
>> very easily make docs into persistent state-machines. And the
>> Comet-style requests mixed with client-specified filters makes it very
>> easy to write event-driven browser apps.
>
> It does indeed. I'm already thinking RSS, IM Server in couchdb
> anybody? The options this might open are rather interesting.
>
I got excited to write this thread when I started drafting out what
IRC would look like if it ran on Couches. :)
Chris
--
Chris Anderson
http://jchris.mfdz.com
Re: all_seqs stream (was Re: Update notifications including update sequence)
Posted by Antony Blakey <an...@gmail.com>.
On 19/01/2009, at 7:05 PM, Ulises wrote:
> That's rather interesting and beg the question whether this wouldn't
> render _external obsolete?
Prompts the question :)
Antony Blakey
-------------
CTO, Linkuistics Pty Ltd
Ph: 0438 840 787
In anything at all, perfection is finally attained not when there is
no longer anything to add, but when there is no longer anything to
take away.
-- Antoine de Saint-Exupery
Re: all_seqs stream (was Re: Update notifications including update sequence)
Posted by Ulises <ul...@gmail.com>.
> The idea behind this is to unify the replication and standard http
> client interfaces for updates streams, so for instance GET replication
> (incrementally updating a db that's current as of seq 200) could work
> via a request like:
>
> /db/_updates?since=200&include_docs=full
>
> And an _external that wanted to see every (or a subset) of updates
> (for instance a Full Text Indexer) could use the same mechanism to
> incrementally maintain an index.
That's rather interesting and beg the question whether this wouldn't
render _external obsolete? You could simply manage your own
ex-_external as a separate process that kept this comet notification
channel open.
> Update streams (especially filtered ones) are one of the most powerful
> tools CouchDB offers. For instance, with filtered streams, you can
> very easily make docs into persistent state-machines. And the
> Comet-style requests mixed with client-specified filters makes it very
> easy to write event-driven browser apps.
It does indeed. I'm already thinking RSS, IM Server in couchdb
anybody? The options this might open are rather interesting.
U
Re: all_seqs stream (was Re: Update notifications including update sequence)
Posted by Chris Anderson <jc...@gmail.com>.
On Mon, Jan 19, 2009 at 12:21 AM, Ulises <ul...@gmail.com> wrote:
>> maybe a better url would be
>>
>> /db/_updates?since=200
>>
>> _updates is a pretty clear name, and easy to read.
>
> Could you not do this with an _external?
>
> (just a thought really)
>
I bet you could implement as an _external.
The idea behind this is to unify the replication and standard http
client interfaces for updates streams, so for instance GET replication
(incrementally updating a db that's current as of seq 200) could work
via a request like:
/db/_updates?since=200&include_docs=full
And an _external that wanted to see every (or a subset) of updates
(for instance a Full Text Indexer) could use the same mechanism to
incrementally maintain an index.
Update streams (especially filtered ones) are one of the most powerful
tools CouchDB offers. For instance, with filtered streams, you can
very easily make docs into persistent state-machines. And the
Comet-style requests mixed with client-specified filters makes it very
easy to write event-driven browser apps.
--
Chris Anderson
http://jchris.mfdz.com
Re: all_seqs stream (was Re: Update notifications including update sequence)
Posted by Ulises <ul...@gmail.com>.
> maybe a better url would be
>
> /db/_updates?since=200
>
> _updates is a pretty clear name, and easy to read.
Could you not do this with an _external?
(just a thought really)
U
Re: all_seqs stream (was Re: Update notifications including update sequence)
Posted by Chris Anderson <jc...@apache.org>.
On Sun, Jan 18, 2009 at 11:55 PM, Chris Anderson <jc...@apache.org> wrote:
> There are some other update-notification features that could be really
> useful, especially for streaming applications.
>
> Damien's called for a Comet-style connection option for replication,
> which brings really cool affordances on the browser (as well as
> industrial strength replication-streams).
>
> Essentially we're talking about an all_seqs view, with various models
> for consuming it.
>
> /db/_all_seqs?since=200
maybe a better url would be
/db/_updates?since=200
_updates is a pretty clear name, and easy to read.
>
> This would send you the updates that happen after seq 200. With this
> you can hold open a connection, waiting for sequence numbers to
> increment.
>
> You can filter the updates
>
> There is also the possibility of filtering the updates with a
> JavaScript (or other) function posted to the server. It returns true
> or false, passing matching updates along to the client. Updates that
> don't match won't be sent to the client. Using this method you can get
> instant updates of eg: my emails.
>
> This is all Damien's magic, basically things he's mentioned on IRC
> etc. I think Couch is ready for us to start refining the client APIs.
> There's room to do some fairly heavy lifting here.
>
> --
> Chris Anderson
> http://jchris.mfdz.com
>
--
Chris Anderson
http://jchris.mfdz.com