You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Ronny Pfannschmidt <Ro...@gmx.de> on 2012/02/28 01:12:15 UTC
feasibility of a design doc option to use the "ddoc new"/"ddoc "
based protocol for map and reduce as well
Hi,
while trying to build a a view server for ddocs that validate/support
documents as FSM (Finite State Machine)
i hit a inherent limit of the protocol,
map and reduce don't get the full ddoc, but only a part of it,
which means my view server can't actually work with the full ddoc
unless i do some weird hacks, and end up in a situation,
where i circumvent proper view invalidation
if map/reduce where allowed to opt in for using the newer protocol for
accessing functions,
my problem would go away
as for view invalidation, a simple variant could just use the _rev,
a more sophisticated one would take a hash of parts of the document
(using excludes/includes defined in options as well)
Regards,
Ronny
Re: feasibility of a design doc option to use the "ddoc new"/"ddoc
" based protocol for map and reduce as well
Posted by Jason Smith <jh...@iriscouch.com>.
I believe the situation is that those functions are not given any
information that could change the results of map() and reduce().
{ "_id": "_design/example"
, "multiply": 50
, "views":
{ "a_view":
{ "map":
"function(D) { emit(D._id, D.value * this.multiply"
}
}
}
Changing .multiply would not invalidate the view signature but it should.
On Tue, Feb 28, 2012 at 9:42 AM, Randall Leeds <ra...@gmail.com> wrote:
> Which part of the document are you missing? I know attachments are not sent
> because that's a lot of overhead for people who attach static resources to
> the design doc. Anything else?
> On Feb 27, 2012 4:12 PM, "Ronny Pfannschmidt" <Ro...@gmx.de>
> wrote:
>
>> Hi,
>>
>> while trying to build a a view server for ddocs that validate/support
>> documents as FSM (Finite State Machine)
>> i hit a inherent limit of the protocol,
>>
>> map and reduce don't get the full ddoc, but only a part of it,
>> which means my view server can't actually work with the full ddoc
>> unless i do some weird hacks, and end up in a situation,
>> where i circumvent proper view invalidation
>>
>> if map/reduce where allowed to opt in for using the newer protocol for
>> accessing functions,
>> my problem would go away
>>
>> as for view invalidation, a simple variant could just use the _rev,
>> a more sophisticated one would take a hash of parts of the document
>> (using excludes/includes defined in options as well)
>>
>> Regards,
>> Ronny
>>
--
Iris Couch
Re: feasibility of a design doc option to use the "ddoc new"/"ddoc
" based protocol for map and reduce as well
Posted by Randall Leeds <ra...@gmail.com>.
Which part of the document are you missing? I know attachments are not sent
because that's a lot of overhead for people who attach static resources to
the design doc. Anything else?
On Feb 27, 2012 4:12 PM, "Ronny Pfannschmidt" <Ro...@gmx.de>
wrote:
> Hi,
>
> while trying to build a a view server for ddocs that validate/support
> documents as FSM (Finite State Machine)
> i hit a inherent limit of the protocol,
>
> map and reduce don't get the full ddoc, but only a part of it,
> which means my view server can't actually work with the full ddoc
> unless i do some weird hacks, and end up in a situation,
> where i circumvent proper view invalidation
>
> if map/reduce where allowed to opt in for using the newer protocol for
> accessing functions,
> my problem would go away
>
> as for view invalidation, a simple variant could just use the _rev,
> a more sophisticated one would take a hash of parts of the document
> (using excludes/includes defined in options as well)
>
> Regards,
> Ronny
>
Re: feasibility of a design doc option to use the "ddoc new"/"ddoc
" based protocol for map and reduce as well
Posted by Ronny Pfannschmidt <Ro...@gmx.de>.
On 02/28/2012 11:24 AM, Benoit Chesneau wrote:
> On Tue, Feb 28, 2012 at 11:09 AM, Jason Smith<jh...@iriscouch.com> wrote:
>> On Tue, Feb 28, 2012 at 10:05 AM, Alexander Shorin<kx...@gmail.com> wrote:
>>> Hi Ronny,
>>>
>>> Invalidating views by ddoc _rev change is very bad idea - your 2M docs
>>> database will have to be reindexed on each ddoc update: by adding
>>> attachment or changing show function. Wait, what's the reason for
>>> views to be invalidated in this case?
>>
>> Ronny, please correct me if I am wrong.
>>
>> But I think the reason is to allow using the *entire* design document
>> to help build views. If so, the _rev invalidation is one thing, but
>> changing CouchDB to send the entire ddoc will be a more substantial
>> change.
>>
>> At any rate, this is why some example failing unit tests might clarify
>> the objective.
>>
>
> why not adding a version property to your ddoc changes ?
>
i started to realize, that a better workaround could actually just
put the data required for my viewservers view handling into the
doc.views.libs attribute
then changes to that would automatically invalidate the views without
breaking everything
i will investigate how to lay out my ddocs to get that behavior
-- Ronny
Re: feasibility of a design doc option to use the "ddoc new"/"ddoc
" based protocol for map and reduce as well
Posted by Benoit Chesneau <bc...@gmail.com>.
On Tue, Feb 28, 2012 at 11:09 AM, Jason Smith <jh...@iriscouch.com> wrote:
> On Tue, Feb 28, 2012 at 10:05 AM, Alexander Shorin <kx...@gmail.com> wrote:
>> Hi Ronny,
>>
>> Invalidating views by ddoc _rev change is very bad idea - your 2M docs
>> database will have to be reindexed on each ddoc update: by adding
>> attachment or changing show function. Wait, what's the reason for
>> views to be invalidated in this case?
>
> Ronny, please correct me if I am wrong.
>
> But I think the reason is to allow using the *entire* design document
> to help build views. If so, the _rev invalidation is one thing, but
> changing CouchDB to send the entire ddoc will be a more substantial
> change.
>
> At any rate, this is why some example failing unit tests might clarify
> the objective.
>
why not adding a version property to your ddoc changes ?
- benoît
Re: feasibility of a design doc option to use the "ddoc new"/"ddoc
" based protocol for map and reduce as well
Posted by Jason Smith <jh...@iriscouch.com>.
On Tue, Feb 28, 2012 at 10:05 AM, Alexander Shorin <kx...@gmail.com> wrote:
> Hi Ronny,
>
> Invalidating views by ddoc _rev change is very bad idea - your 2M docs
> database will have to be reindexed on each ddoc update: by adding
> attachment or changing show function. Wait, what's the reason for
> views to be invalidated in this case?
Ronny, please correct me if I am wrong.
But I think the reason is to allow using the *entire* design document
to help build views. If so, the _rev invalidation is one thing, but
changing CouchDB to send the entire ddoc will be a more substantial
change.
At any rate, this is why some example failing unit tests might clarify
the objective.
--
Iris Couch
Re: feasibility of a design doc option to use the "ddoc new"/"ddoc
" based protocol for map and reduce as well
Posted by Alexander Shorin <kx...@gmail.com>.
Hi Ronny,
Invalidating views by ddoc _rev change is very bad idea - your 2M docs
database will have to be reindexed on each ddoc update: by adding
attachment or changing show function. Wait, what's the reason for
views to be invalidated in this case?
--
,,,^..^,,,
On Tue, Feb 28, 2012 at 12:45 PM, Ronny Pfannschmidt
<Ro...@gmx.de> wrote:
> On 02/28/2012 04:09 AM, Jason Smith wrote:
>>
>> On Tue, Feb 28, 2012 at 7:12 AM, Ronny Pfannschmidt
>> <Ro...@gmx.de> wrote:
>>>
>>> Hi,
>>>
>>> while trying to build a a view server for ddocs that validate/support
>>> documents as FSM (Finite State Machine)
>>> i hit a inherent limit of the protocol,
>>>
>>> map and reduce don't get the full ddoc, but only a part of it,
>>> which means my view server can't actually work with the full ddoc
>>> unless i do some weird hacks, and end up in a situation,
>>> where i circumvent proper view invalidation
>>>
>>> if map/reduce where allowed to opt in for using the newer protocol for
>>> accessing functions,
>>> my problem would go away
>>>
>>> as for view invalidation, a simple variant could just use the _rev,
>>> a more sophisticated one would take a hash of parts of the document
>>> (using excludes/includes defined in options as well)
>>
>>
>> Hi, Ronny. Are you aware that the contents of .views.lib are sent to
>> the view server? At least with Javascript, the idea is that CommonJS
>> modules can go in there.
>>
>> Maybe that can help as a workaround for now.
>>
>
> Hi Jason,
>
> rather than just a workaround,
> i would like to know the likelihood of accepting a patch that implements the
> view option + using _rev as invalidation hint
>
> also i cant find docs on the protocol that's being used for exchanging
> CommonJS of views to the viewserver
>
> -- Ronny
Re: feasibility of a design doc option to use the "ddoc new"/"ddoc
" based protocol for map and reduce as well
Posted by Ronny Pfannschmidt <Ro...@gmx.de>.
On 02/28/2012 04:09 AM, Jason Smith wrote:
> On Tue, Feb 28, 2012 at 7:12 AM, Ronny Pfannschmidt
> <Ro...@gmx.de> wrote:
>> Hi,
>>
>> while trying to build a a view server for ddocs that validate/support
>> documents as FSM (Finite State Machine)
>> i hit a inherent limit of the protocol,
>>
>> map and reduce don't get the full ddoc, but only a part of it,
>> which means my view server can't actually work with the full ddoc
>> unless i do some weird hacks, and end up in a situation,
>> where i circumvent proper view invalidation
>>
>> if map/reduce where allowed to opt in for using the newer protocol for
>> accessing functions,
>> my problem would go away
>>
>> as for view invalidation, a simple variant could just use the _rev,
>> a more sophisticated one would take a hash of parts of the document
>> (using excludes/includes defined in options as well)
>
> Hi, Ronny. Are you aware that the contents of .views.lib are sent to
> the view server? At least with Javascript, the idea is that CommonJS
> modules can go in there.
>
> Maybe that can help as a workaround for now.
>
Hi Jason,
rather than just a workaround,
i would like to know the likelihood of accepting a patch that implements
the view option + using _rev as invalidation hint
also i cant find docs on the protocol that's being used for exchanging
CommonJS of views to the viewserver
-- Ronny
Re: feasibility of a design doc option to use the "ddoc new"/"ddoc
" based protocol for map and reduce as well
Posted by Jason Smith <jh...@iriscouch.com>.
On Tue, Feb 28, 2012 at 7:12 AM, Ronny Pfannschmidt
<Ro...@gmx.de> wrote:
> Hi,
>
> while trying to build a a view server for ddocs that validate/support
> documents as FSM (Finite State Machine)
> i hit a inherent limit of the protocol,
>
> map and reduce don't get the full ddoc, but only a part of it,
> which means my view server can't actually work with the full ddoc
> unless i do some weird hacks, and end up in a situation,
> where i circumvent proper view invalidation
>
> if map/reduce where allowed to opt in for using the newer protocol for
> accessing functions,
> my problem would go away
>
> as for view invalidation, a simple variant could just use the _rev,
> a more sophisticated one would take a hash of parts of the document
> (using excludes/includes defined in options as well)
Hi, Ronny. Are you aware that the contents of .views.lib are sent to
the view server? At least with Javascript, the idea is that CommonJS
modules can go in there.
Maybe that can help as a workaround for now.
--
Iris Couch