You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Jedediah Smith <je...@silencegreys.com> on 2008/11/18 20:30:29 UTC
Re: View with incremental counter
Grasping the concepts involved in map/reduce might end up being a
significant hurdle to CouchDB adoption so I would encourage features
that assist in that regard. Sandboxing might also be necessary to
enforce whatever is ultimately CouchDB's security model.
Spidermonkey must have sandboxing capabilities if it runs the javascript
for both the Firefox browser and the web pages it loads.
Paul's case of caching large objects is something to consider (and a
reason my previous email was not strictly correct) but could perhaps be
handled more formally or left to custom view servers.
Benjamin Nortier wrote:
> Should CouchDB not restrict defining state outside the map function?
> I.e. you should not be able to declare a counter outside the function?
>
> On Tue, Nov 18, 2008 at 1:42 PM, Jedediah Smith
> <je...@silencegreys.com> wrote:
>> You can't maintain a state across calls to map functions in this way. Map
>> functions can be called in any order or in parallel, any number of times on
>> a particular document and in completely separate environments. They should
>> not have any side effects or depend on any outside state.
>>
>> You should read up on Google's MapReduce and maybe functional programming in
>> general in order to understand how CouchDB works.
>>
>> If you just want to number the results of your view, that can be easily done
>> by the calling code.
>>
>> Adam Groves wrote:
>>> Hi,
>>>
>>> I've got the following view:
>>>
>>> count = 0;
>>> function(doc) {
>>> if(doc.type == "document") {
>>> count = count + 1
>>> emit(doc._id, count);
>>> }
>>> }
>>>
>>> The idea is that I get a list of documents, each having a counter
>>> value which is incremental. I expected the count values to come out in
>>> order, but that isn't the case: my first few documents have values of
>>> 62, 61, 22, 19. I'm not quite sure what's going on here - any ideas
>>> how the order is being set here?
>>>
>>> Regards
>>>
>>> Adam Groves
>>>
>
>
>
Re: View with incremental counter
Posted by Chris Anderson <jc...@apache.org>.
> Ben Browning wrote:
>>
>> I have several views that actually include some javascript libraries
>> inline within the view outside of my map function (date.js for
>> example).
I've been doing that also, and it seems to work pretty well. I haven't
benchmarked to see if the extra code slows it down, but I know giant
map functions don't break anything. (I tend to include my inline code
within my view function, but I expect Spidermonkey can still optimize
it.)
--
Chris Anderson
http://jchris.mfdz.com
Re: View with incremental counter
Posted by Jedediah Smith <je...@silencegreys.com>.
Perhaps design docs could be allowed to provide init functions which
would be passed to the view server whenever the view server is
(re)started or the design doc is updated?
The view server would maintain a distinct, persistent sandbox for each
design doc (or however else you want to partition it) and the id of said
design doc would have to be passed with view protocol calls, if it isn't
already.
To enforce the map/reduce semantic requirements, the view server could
make the environment immutable for map/reduce calls and unlock it for
init calls.
Ben Browning wrote:
> I have several views that actually include some javascript libraries
> inline within the view outside of my map function (date.js for
> example). So, hopefully any sort of sandbox functionality would take
> this use-case into account.
>
> - Ben
>
Re: View with incremental counter
Posted by Ben Browning <be...@gmail.com>.
I have several views that actually include some javascript libraries
inline within the view outside of my map function (date.js for
example). So, hopefully any sort of sandbox functionality would take
this use-case into account.
- Ben
Re: View with incremental counter
Posted by Damien Katz <da...@apache.org>.
That, and sealing the context too, to prevent these sorts of things.
-Damien
On Nov 18, 2008, at 2:45 PM, Paul Davis wrote:
> Wasn't that the document sealing to prevent maps from interfering with
> each other?
>
> On Tue, Nov 18, 2008 at 2:35 PM, Damien Katz <da...@apache.org>
> wrote:
>> Spidermonkey's sandboxing capabilities vary release to release. As
>> I recall,
>> we had full sandboxing working with earlier releases, but something
>> changed
>> in 1.7 and we took it out.
>>
>> -Damien
>>
>> On Nov 18, 2008, at 2:30 PM, Jedediah Smith wrote:
>>
>>>
>>> Grasping the concepts involved in map/reduce might end up being a
>>> significant hurdle to CouchDB adoption so I would encourage
>>> features that
>>> assist in that regard. Sandboxing might also be necessary to enforce
>>> whatever is ultimately CouchDB's security model.
>>>
>>> Spidermonkey must have sandboxing capabilities if it runs the
>>> javascript
>>> for both the Firefox browser and the web pages it loads.
>>>
>>> Paul's case of caching large objects is something to consider (and a
>>> reason my previous email was not strictly correct) but could
>>> perhaps be
>>> handled more formally or left to custom view servers.
>>>
>>> Benjamin Nortier wrote:
>>>>
>>>> Should CouchDB not restrict defining state outside the map
>>>> function?
>>>> I.e. you should not be able to declare a counter outside the
>>>> function?
>>>> On Tue, Nov 18, 2008 at 1:42 PM, Jedediah Smith
>>>> <je...@silencegreys.com> wrote:
>>>>>
>>>>> You can't maintain a state across calls to map functions in this
>>>>> way.
>>>>> Map
>>>>> functions can be called in any order or in parallel, any number
>>>>> of times
>>>>> on
>>>>> a particular document and in completely separate environments.
>>>>> They
>>>>> should
>>>>> not have any side effects or depend on any outside state.
>>>>>
>>>>> You should read up on Google's MapReduce and maybe functional
>>>>> programming in
>>>>> general in order to understand how CouchDB works.
>>>>>
>>>>> If you just want to number the results of your view, that can be
>>>>> easily
>>>>> done
>>>>> by the calling code.
>>>>>
>>>>> Adam Groves wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I've got the following view:
>>>>>>
>>>>>> count = 0;
>>>>>> function(doc) {
>>>>>> if(doc.type == "document") {
>>>>>> count = count + 1
>>>>>> emit(doc._id, count);
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> The idea is that I get a list of documents, each having a counter
>>>>>> value which is incremental. I expected the count values to come
>>>>>> out in
>>>>>> order, but that isn't the case: my first few documents have
>>>>>> values of
>>>>>> 62, 61, 22, 19. I'm not quite sure what's going on here - any
>>>>>> ideas
>>>>>> how the order is being set here?
>>>>>>
>>>>>> Regards
>>>>>>
>>>>>> Adam Groves
>>>>>>
>>
>>
Re: View with incremental counter
Posted by Paul Davis <pa...@gmail.com>.
Wasn't that the document sealing to prevent maps from interfering with
each other?
On Tue, Nov 18, 2008 at 2:35 PM, Damien Katz <da...@apache.org> wrote:
> Spidermonkey's sandboxing capabilities vary release to release. As I recall,
> we had full sandboxing working with earlier releases, but something changed
> in 1.7 and we took it out.
>
> -Damien
>
> On Nov 18, 2008, at 2:30 PM, Jedediah Smith wrote:
>
>>
>> Grasping the concepts involved in map/reduce might end up being a
>> significant hurdle to CouchDB adoption so I would encourage features that
>> assist in that regard. Sandboxing might also be necessary to enforce
>> whatever is ultimately CouchDB's security model.
>>
>> Spidermonkey must have sandboxing capabilities if it runs the javascript
>> for both the Firefox browser and the web pages it loads.
>>
>> Paul's case of caching large objects is something to consider (and a
>> reason my previous email was not strictly correct) but could perhaps be
>> handled more formally or left to custom view servers.
>>
>> Benjamin Nortier wrote:
>>>
>>> Should CouchDB not restrict defining state outside the map function?
>>> I.e. you should not be able to declare a counter outside the function?
>>> On Tue, Nov 18, 2008 at 1:42 PM, Jedediah Smith
>>> <je...@silencegreys.com> wrote:
>>>>
>>>> You can't maintain a state across calls to map functions in this way.
>>>> Map
>>>> functions can be called in any order or in parallel, any number of times
>>>> on
>>>> a particular document and in completely separate environments. They
>>>> should
>>>> not have any side effects or depend on any outside state.
>>>>
>>>> You should read up on Google's MapReduce and maybe functional
>>>> programming in
>>>> general in order to understand how CouchDB works.
>>>>
>>>> If you just want to number the results of your view, that can be easily
>>>> done
>>>> by the calling code.
>>>>
>>>> Adam Groves wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I've got the following view:
>>>>>
>>>>> count = 0;
>>>>> function(doc) {
>>>>> if(doc.type == "document") {
>>>>> count = count + 1
>>>>> emit(doc._id, count);
>>>>> }
>>>>> }
>>>>>
>>>>> The idea is that I get a list of documents, each having a counter
>>>>> value which is incremental. I expected the count values to come out in
>>>>> order, but that isn't the case: my first few documents have values of
>>>>> 62, 61, 22, 19. I'm not quite sure what's going on here - any ideas
>>>>> how the order is being set here?
>>>>>
>>>>> Regards
>>>>>
>>>>> Adam Groves
>>>>>
>
>
Re: View with incremental counter
Posted by Damien Katz <da...@apache.org>.
Spidermonkey's sandboxing capabilities vary release to release. As I
recall, we had full sandboxing working with earlier releases, but
something changed in 1.7 and we took it out.
-Damien
On Nov 18, 2008, at 2:30 PM, Jedediah Smith wrote:
>
> Grasping the concepts involved in map/reduce might end up being a
> significant hurdle to CouchDB adoption so I would encourage features
> that assist in that regard. Sandboxing might also be necessary to
> enforce whatever is ultimately CouchDB's security model.
>
> Spidermonkey must have sandboxing capabilities if it runs the
> javascript for both the Firefox browser and the web pages it loads.
>
> Paul's case of caching large objects is something to consider (and a
> reason my previous email was not strictly correct) but could perhaps
> be handled more formally or left to custom view servers.
>
> Benjamin Nortier wrote:
>> Should CouchDB not restrict defining state outside the map function?
>> I.e. you should not be able to declare a counter outside the
>> function?
>> On Tue, Nov 18, 2008 at 1:42 PM, Jedediah Smith
>> <je...@silencegreys.com> wrote:
>>> You can't maintain a state across calls to map functions in this
>>> way. Map
>>> functions can be called in any order or in parallel, any number of
>>> times on
>>> a particular document and in completely separate environments.
>>> They should
>>> not have any side effects or depend on any outside state.
>>>
>>> You should read up on Google's MapReduce and maybe functional
>>> programming in
>>> general in order to understand how CouchDB works.
>>>
>>> If you just want to number the results of your view, that can be
>>> easily done
>>> by the calling code.
>>>
>>> Adam Groves wrote:
>>>> Hi,
>>>>
>>>> I've got the following view:
>>>>
>>>> count = 0;
>>>> function(doc) {
>>>> if(doc.type == "document") {
>>>> count = count + 1
>>>> emit(doc._id, count);
>>>> }
>>>> }
>>>>
>>>> The idea is that I get a list of documents, each having a counter
>>>> value which is incremental. I expected the count values to come
>>>> out in
>>>> order, but that isn't the case: my first few documents have
>>>> values of
>>>> 62, 61, 22, 19. I'm not quite sure what's going on here - any ideas
>>>> how the order is being set here?
>>>>
>>>> Regards
>>>>
>>>> Adam Groves
>>>>
Re: View with incremental counter
Posted by Damien Katz <da...@apache.org>.
Spidermonkey's sandboxing capabilities vary release to release. As I
recall, we had full sandboxing working with earlier releases, but
something changed in 1.7 and we took it out.
-Damien
On Nov 18, 2008, at 2:30 PM, Jedediah Smith wrote:
>
> Grasping the concepts involved in map/reduce might end up being a
> significant hurdle to CouchDB adoption so I would encourage features
> that assist in that regard. Sandboxing might also be necessary to
> enforce whatever is ultimately CouchDB's security model.
>
> Spidermonkey must have sandboxing capabilities if it runs the
> javascript for both the Firefox browser and the web pages it loads.
>
> Paul's case of caching large objects is something to consider (and a
> reason my previous email was not strictly correct) but could perhaps
> be handled more formally or left to custom view servers.
>
> Benjamin Nortier wrote:
>> Should CouchDB not restrict defining state outside the map function?
>> I.e. you should not be able to declare a counter outside the
>> function?
>> On Tue, Nov 18, 2008 at 1:42 PM, Jedediah Smith
>> <je...@silencegreys.com> wrote:
>>> You can't maintain a state across calls to map functions in this
>>> way. Map
>>> functions can be called in any order or in parallel, any number of
>>> times on
>>> a particular document and in completely separate environments.
>>> They should
>>> not have any side effects or depend on any outside state.
>>>
>>> You should read up on Google's MapReduce and maybe functional
>>> programming in
>>> general in order to understand how CouchDB works.
>>>
>>> If you just want to number the results of your view, that can be
>>> easily done
>>> by the calling code.
>>>
>>> Adam Groves wrote:
>>>> Hi,
>>>>
>>>> I've got the following view:
>>>>
>>>> count = 0;
>>>> function(doc) {
>>>> if(doc.type == "document") {
>>>> count = count + 1
>>>> emit(doc._id, count);
>>>> }
>>>> }
>>>>
>>>> The idea is that I get a list of documents, each having a counter
>>>> value which is incremental. I expected the count values to come
>>>> out in
>>>> order, but that isn't the case: my first few documents have
>>>> values of
>>>> 62, 61, 22, 19. I'm not quite sure what's going on here - any ideas
>>>> how the order is being set here?
>>>>
>>>> Regards
>>>>
>>>> Adam Groves
>>>>