You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Norman Rosner <no...@googlemail.com> on 2010/02/17 21:04:19 UTC
Chaining of views/MapReduce
Hi folks,
first, I'll have to admit that I'm kinda new to JavaScript and of course to CouchDB. Second I just reuse the subject so I hope it also pops up if anybody searches for it.
As I read chaining of views is not possible yet but it's mentioned couple of times on the mailing list. So here's what I want to do:
1. Create a list of unique labels/tags/whatever through all of the documents (e.g. all nouns that are in the documents)
2. Extract all labels/tags/nouns of each document and check them again the before calculated result in some kind of way
For the second point I created a view which works except of the checking against the result from point 1. Now I'm trying to solve point 1.
And here my questions begin: How can I create/update a document from inside of a map function? As I think of it, I'll have to make a HTTP GET to load the document in each iteration. I found some HTTP stuff in the test.js in the test folder, but I'm not quite sure how to use it and if it's the right way of thinking? Is there any way of using global variables throughout 'couchapps' (e.g. through the lib folder and thelike)?
Any help of you CouchDB kings would be greatly appreciated!
Cheers,
norman
Re: Chaining of views/MapReduce
Posted by Mario Scheliga <ma...@sourcegarden.de>.
Hi Norman,
ahhh, the light is shining bright now. So the Result of Map/Phase-1 is
the input of Map/Phase-2? I guess this would fit in couchdb and its a
great feature, but where to store the intermediate-results in the b-
tree for caching purposes?
Perhaps there have to be another kind of query-server to solve this?
Because the intermediate-results dont have to be cached, right?
cheers,
mario
Am 18.02.2010 um 11:20 schrieb Norman Rosner:
> Hi Mario,
>
> you're probably right about the map phase and read only and all
> that. Hovercraft doesn't look right to me though. I found another
> thread on the mailinglist (http://markmail.org/thread/
> ne2ghnwpojxkhalj) that handles a similar topic I guess.
>
> In Hadoop you can chain jobs (map/reduce phases). So you can take
> the output of the first job as the input of the second job and so
> on. But Hadoop is based on a distributed filesystem, so the results
> are merged together into one location after a job is done, so you
> don't have to think about the thousand servers ;)
>
> I'll guess I will write a workaround in Java to pipe in all the rows
> of the view, extract my stuff, cache it somehow and write it back as
> a new document to the database. After that I could check my rows of
> the view against the newly created doc.
>
> Cheers,
> norman
> On 18.02.2010, at 09:01, Mario Scheliga wrote:
>
>> Hi Norman,
>>
>> i think its obvious that this wont be possible with couchdb itself.
>> but i think hovercraft by jchris can do that for you.
>>
>> http://github.com/jchris/hovercraft
>>
>> otherwise you have to implement the second check after re-reduce on
>> the client side.
>> because, map-function only read data and create new ones (in the
>> map only).
>> writing docs is left to the other processes (put docs or update).
>>
>> could you explain it for me, how you do this in hadoop?
>>
>> greetz
>> mario
>>
>> Am 17.02.2010 um 23:29 schrieb Norman Rosner:
>>
>>>
>>> On 17.02.2010, at 23:15, Mario Scheliga wrote:
>>>
>>>> Hi Norman,
>>>>
>>>> updating a document from map-function its not possible and seems
>>>> to be the wrong way.
>>>> Thinking of map function processing docs seperatly (sandbox), so
>>>> you are able to
>>>> spread the execution over thousand of servers ;-)
>>>
>>> True that! But: suppose I'm just creating/updating one document
>>> per couchdb-instance, that should be ok, right? Because after
>>> that, I can easily get all the result documents and merge them
>>> together. I would do it in as similar way in Hadoop. And as far as
>>> I read in the loooong archives of this list, I'm not the only one
>>> who wants to do such things.
>>>
>>> cheers,
>>> norman
>>>
>>>>
>>>> greez
>>>> mario
>>>>
>>>> Am 17.02.2010 um 21:04 schrieb Norman Rosner:
>>>>
>>>>> Hi folks,
>>>>>
>>>>> first, I'll have to admit that I'm kinda new to JavaScript and
>>>>> of course to CouchDB. Second I just reuse the subject so I hope
>>>>> it also pops up if anybody searches for it.
>>>>>
>>>>> As I read chaining of views is not possible yet but it's
>>>>> mentioned couple of times on the mailing list. So here's what I
>>>>> want to do:
>>>>>
>>>>> 1. Create a list of unique labels/tags/whatever through all of
>>>>> the documents (e.g. all nouns that are in the documents)
>>>>> 2. Extract all labels/tags/nouns of each document and check them
>>>>> again the before calculated result in some kind of way
>>>>>
>>>>> For the second point I created a view which works except of the
>>>>> checking against the result from point 1. Now I'm trying to
>>>>> solve point 1.
>>>>> And here my questions begin: How can I create/update a document
>>>>> from inside of a map function? As I think of it, I'll have to
>>>>> make a HTTP GET to load the document in each iteration. I found
>>>>> some HTTP stuff in the test.js in the test folder, but I'm not
>>>>> quite sure how to use it and if it's the right way of thinking?
>>>>> Is there any way of using global variables throughout
>>>>> 'couchapps' (e.g. through the lib folder and thelike)?
>>>>>
>>>>> Any help of you CouchDB kings would be greatly appreciated!
>>>>>
>>>>> Cheers,
>>>>>
>>>>> norman
>>>>
>>>>
>>>> --
>>>> Sourcegarden GmbH HR: B-104357
>>>> Steuernummer: 37/167/21214 USt-ID: DE814784953
>>>> Geschaeftsfuehrer: Mario Scheliga, Rene Otto
>>>> Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
>>>> Schoenhauser Allee 51, 10437 Berlin
>>>>
>>>
>>
>>
>> --
>> Sourcegarden GmbH HR: B-104357
>> Steuernummer: 37/167/21214 USt-ID: DE814784953
>> Geschaeftsfuehrer: Mario Scheliga, Rene Otto
>> Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
>> Schoenhauser Allee 51, 10437 Berlin
>>
>
--
Sourcegarden GmbH HR: B-104357
Steuernummer: 37/167/21214 USt-ID: DE814784953
Geschaeftsfuehrer: Mario Scheliga, Rene Otto
Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
Schoenhauser Allee 51, 10437 Berlin
Re: Chaining of views/MapReduce
Posted by Mario Scheliga <ma...@sourcegarden.de>.
i guess the update-functions doesn't check this at all. (in 0.10.0)
Am 19.02.2010 um 13:50 schrieb Markus Jelsma:
> all-or-nothing - To use this mode, include "all_or_nothing":true as
> part of
> the request. In the case of a power failure, when the database
> restarts either
> all the changes will have been saved or none of them. However, it
> does not do
> conflict checking, so the documents will be committed even if this
> creates
> conflicts.
>
>
> [1] http://wiki.apache.org/couchdb/HTTP_Bulk_Document_API
>
>
>
> Markus Jelsma - Technisch Architect - Buyways BV
> http://www.linkedin.com/in/markus17
> 050-8536620 / 06-50258350
>
--
Sourcegarden GmbH HR: B-104357
Steuernummer: 37/167/21214 USt-ID: DE814784953
Geschaeftsfuehrer: Mario Scheliga, Rene Otto
Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
Schoenhauser Allee 51, 10437 Berlin
Re: Chaining of views/MapReduce
Posted by Markus Jelsma <ma...@buyways.nl>.
all-or-nothing - To use this mode, include "all_or_nothing":true as part of
the request. In the case of a power failure, when the database restarts either
all the changes will have been saved or none of them. However, it does not do
conflict checking, so the documents will be committed even if this creates
conflicts.
[1] http://wiki.apache.org/couchdb/HTTP_Bulk_Document_API
Markus Jelsma - Technisch Architect - Buyways BV
http://www.linkedin.com/in/markus17
050-8536620 / 06-50258350
Re: Chaining of views/MapReduce
Posted by Mario Scheliga <ma...@sourcegarden.de>.
Ahh okay,
why you have to keep track of the revision!?
its possible to create/update documents without revision-check, would
that help?
greetz
mario
Am 19.02.2010 um 13:25 schrieb Norman Rosner:
> Mario!
>
> I read that thread as well. But still you need to be able to make
> HTTP calls inside of the map or reduce function. Since my JavaScript
> skills are limited ;) I'm not able to write something like that (And
> right now I don't have the time to learn it in depth right now). So
> I found this JS HTTP wrapper in the test.js file from the test suite
> of couchdb. But anyways I haven't been able to use it (<- limited JS
> skills ;) )
>
> In my opinion the map function is read only to the document it
> handles, which totally makes sense. But you should be able to create
> new documents. It should be ok to write those documents back to the
> original db. (http://bit.ly/dvnnJH)
> But like I said, you need to make HTTP requests inside of the map/
> reduce function. And in my situation you have to keep track of the
> revision of the document. So you need to come up with some variable
> that's accessable through all the maps...
>
> Cheers,
>
> norman
>
> On 19.02.2010, at 11:31, Mario Scheliga wrote:
>
>> Hi Norman,
>>
>> i found a post by P.Davis that contains a possibility
>>
>> http://mail-archives.apache.org/mod_mbox/couchdb-user/200910.mbox/%3Ce2111bbb0910280610q37de08ack316c628a4a48cbcc@mail.gmail.com%3E
>>
>> chaining could be done by some serverside logic, where the result
>> is stored in a seperate db.
>> what do you think?
>>
>> greetz
>> mario
>>
>> Am 18.02.2010 um 11:20 schrieb Norman Rosner:
>>
>>> Hi Mario,
>>>
>>> you're probably right about the map phase and read only and all
>>> that. Hovercraft doesn't look right to me though. I found another
>>> thread on the mailinglist (http://markmail.org/thread/ne2ghnwpojxkhalj
>>> ) that handles a similar topic I guess.
>>>
>>> In Hadoop you can chain jobs (map/reduce phases). So you can take
>>> the output of the first job as the input of the second job and so
>>> on. But Hadoop is based on a distributed filesystem, so the
>>> results are merged together into one location after a job is done,
>>> so you don't have to think about the thousand servers ;)
>>>
>>> I'll guess I will write a workaround in Java to pipe in all the
>>> rows of the view, extract my stuff, cache it somehow and write it
>>> back as a new document to the database. After that I could check
>>> my rows of the view against the newly created doc.
>>>
>>> Cheers,
>>> norman
>>> On 18.02.2010, at 09:01, Mario Scheliga wrote:
>>>
>>>> Hi Norman,
>>>>
>>>> i think its obvious that this wont be possible with couchdb
>>>> itself. but i think hovercraft by jchris can do that for you.
>>>>
>>>> http://github.com/jchris/hovercraft
>>>>
>>>> otherwise you have to implement the second check after re-reduce
>>>> on the client side.
>>>> because, map-function only read data and create new ones (in the
>>>> map only).
>>>> writing docs is left to the other processes (put docs or update).
>>>>
>>>> could you explain it for me, how you do this in hadoop?
>>>>
>>>> greetz
>>>> mario
>>>>
>>>> Am 17.02.2010 um 23:29 schrieb Norman Rosner:
>>>>
>>>>>
>>>>> On 17.02.2010, at 23:15, Mario Scheliga wrote:
>>>>>
>>>>>> Hi Norman,
>>>>>>
>>>>>> updating a document from map-function its not possible and
>>>>>> seems to be the wrong way.
>>>>>> Thinking of map function processing docs seperatly (sandbox),
>>>>>> so you are able to
>>>>>> spread the execution over thousand of servers ;-)
>>>>>
>>>>> True that! But: suppose I'm just creating/updating one document
>>>>> per couchdb-instance, that should be ok, right? Because after
>>>>> that, I can easily get all the result documents and merge them
>>>>> together. I would do it in as similar way in Hadoop. And as far
>>>>> as I read in the loooong archives of this list, I'm not the only
>>>>> one who wants to do such things.
>>>>>
>>>>> cheers,
>>>>> norman
>>>>>
>>>>>>
>>>>>> greez
>>>>>> mario
>>>>>>
>>>>>> Am 17.02.2010 um 21:04 schrieb Norman Rosner:
>>>>>>
>>>>>>> Hi folks,
>>>>>>>
>>>>>>> first, I'll have to admit that I'm kinda new to JavaScript and
>>>>>>> of course to CouchDB. Second I just reuse the subject so I
>>>>>>> hope it also pops up if anybody searches for it.
>>>>>>>
>>>>>>> As I read chaining of views is not possible yet but it's
>>>>>>> mentioned couple of times on the mailing list. So here's what
>>>>>>> I want to do:
>>>>>>>
>>>>>>> 1. Create a list of unique labels/tags/whatever through all of
>>>>>>> the documents (e.g. all nouns that are in the documents)
>>>>>>> 2. Extract all labels/tags/nouns of each document and check
>>>>>>> them again the before calculated result in some kind of way
>>>>>>>
>>>>>>> For the second point I created a view which works except of
>>>>>>> the checking against the result from point 1. Now I'm trying
>>>>>>> to solve point 1.
>>>>>>> And here my questions begin: How can I create/update a
>>>>>>> document from inside of a map function? As I think of it, I'll
>>>>>>> have to make a HTTP GET to load the document in each
>>>>>>> iteration. I found some HTTP stuff in the test.js in the test
>>>>>>> folder, but I'm not quite sure how to use it and if it's the
>>>>>>> right way of thinking? Is there any way of using global
>>>>>>> variables throughout 'couchapps' (e.g. through the lib folder
>>>>>>> and thelike)?
>>>>>>>
>>>>>>> Any help of you CouchDB kings would be greatly appreciated!
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> norman
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Sourcegarden GmbH HR: B-104357
>>>>>> Steuernummer: 37/167/21214 USt-ID: DE814784953
>>>>>> Geschaeftsfuehrer: Mario Scheliga, Rene Otto
>>>>>> Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
>>>>>> Schoenhauser Allee 51, 10437 Berlin
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Sourcegarden GmbH HR: B-104357
>>>> Steuernummer: 37/167/21214 USt-ID: DE814784953
>>>> Geschaeftsfuehrer: Mario Scheliga, Rene Otto
>>>> Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
>>>> Schoenhauser Allee 51, 10437 Berlin
>>>>
>>>
>>
>>
>> --
>> Sourcegarden GmbH HR: B-104357
>> Steuernummer: 37/167/21214 USt-ID: DE814784953
>> Geschaeftsfuehrer: Mario Scheliga, Rene Otto
>> Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
>> Schoenhauser Allee 51, 10437 Berlin
>>
>
--
Sourcegarden GmbH HR: B-104357
Steuernummer: 37/167/21214 USt-ID: DE814784953
Geschaeftsfuehrer: Mario Scheliga, Rene Otto
Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
Schoenhauser Allee 51, 10437 Berlin
Re: Chaining of views/MapReduce
Posted by Norman Rosner <no...@googlemail.com>.
Mario!
I read that thread as well. But still you need to be able to make HTTP calls inside of the map or reduce function. Since my JavaScript skills are limited ;) I'm not able to write something like that (And right now I don't have the time to learn it in depth right now). So I found this JS HTTP wrapper in the test.js file from the test suite of couchdb. But anyways I haven't been able to use it (<- limited JS skills ;) )
In my opinion the map function is read only to the document it handles, which totally makes sense. But you should be able to create new documents. It should be ok to write those documents back to the original db. (http://bit.ly/dvnnJH)
But like I said, you need to make HTTP requests inside of the map/reduce function. And in my situation you have to keep track of the revision of the document. So you need to come up with some variable that's accessable through all the maps...
Cheers,
norman
On 19.02.2010, at 11:31, Mario Scheliga wrote:
> Hi Norman,
>
> i found a post by P.Davis that contains a possibility
>
> http://mail-archives.apache.org/mod_mbox/couchdb-user/200910.mbox/%3Ce2111bbb0910280610q37de08ack316c628a4a48cbcc@mail.gmail.com%3E
>
> chaining could be done by some serverside logic, where the result is stored in a seperate db.
> what do you think?
>
> greetz
> mario
>
> Am 18.02.2010 um 11:20 schrieb Norman Rosner:
>
>> Hi Mario,
>>
>> you're probably right about the map phase and read only and all that. Hovercraft doesn't look right to me though. I found another thread on the mailinglist (http://markmail.org/thread/ne2ghnwpojxkhalj) that handles a similar topic I guess.
>>
>> In Hadoop you can chain jobs (map/reduce phases). So you can take the output of the first job as the input of the second job and so on. But Hadoop is based on a distributed filesystem, so the results are merged together into one location after a job is done, so you don't have to think about the thousand servers ;)
>>
>> I'll guess I will write a workaround in Java to pipe in all the rows of the view, extract my stuff, cache it somehow and write it back as a new document to the database. After that I could check my rows of the view against the newly created doc.
>>
>> Cheers,
>> norman
>> On 18.02.2010, at 09:01, Mario Scheliga wrote:
>>
>>> Hi Norman,
>>>
>>> i think its obvious that this wont be possible with couchdb itself. but i think hovercraft by jchris can do that for you.
>>>
>>> http://github.com/jchris/hovercraft
>>>
>>> otherwise you have to implement the second check after re-reduce on the client side.
>>> because, map-function only read data and create new ones (in the map only).
>>> writing docs is left to the other processes (put docs or update).
>>>
>>> could you explain it for me, how you do this in hadoop?
>>>
>>> greetz
>>> mario
>>>
>>> Am 17.02.2010 um 23:29 schrieb Norman Rosner:
>>>
>>>>
>>>> On 17.02.2010, at 23:15, Mario Scheliga wrote:
>>>>
>>>>> Hi Norman,
>>>>>
>>>>> updating a document from map-function its not possible and seems to be the wrong way.
>>>>> Thinking of map function processing docs seperatly (sandbox), so you are able to
>>>>> spread the execution over thousand of servers ;-)
>>>>
>>>> True that! But: suppose I'm just creating/updating one document per couchdb-instance, that should be ok, right? Because after that, I can easily get all the result documents and merge them together. I would do it in as similar way in Hadoop. And as far as I read in the loooong archives of this list, I'm not the only one who wants to do such things.
>>>>
>>>> cheers,
>>>> norman
>>>>
>>>>>
>>>>> greez
>>>>> mario
>>>>>
>>>>> Am 17.02.2010 um 21:04 schrieb Norman Rosner:
>>>>>
>>>>>> Hi folks,
>>>>>>
>>>>>> first, I'll have to admit that I'm kinda new to JavaScript and of course to CouchDB. Second I just reuse the subject so I hope it also pops up if anybody searches for it.
>>>>>>
>>>>>> As I read chaining of views is not possible yet but it's mentioned couple of times on the mailing list. So here's what I want to do:
>>>>>>
>>>>>> 1. Create a list of unique labels/tags/whatever through all of the documents (e.g. all nouns that are in the documents)
>>>>>> 2. Extract all labels/tags/nouns of each document and check them again the before calculated result in some kind of way
>>>>>>
>>>>>> For the second point I created a view which works except of the checking against the result from point 1. Now I'm trying to solve point 1.
>>>>>> And here my questions begin: How can I create/update a document from inside of a map function? As I think of it, I'll have to make a HTTP GET to load the document in each iteration. I found some HTTP stuff in the test.js in the test folder, but I'm not quite sure how to use it and if it's the right way of thinking? Is there any way of using global variables throughout 'couchapps' (e.g. through the lib folder and thelike)?
>>>>>>
>>>>>> Any help of you CouchDB kings would be greatly appreciated!
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> norman
>>>>>
>>>>>
>>>>> --
>>>>> Sourcegarden GmbH HR: B-104357
>>>>> Steuernummer: 37/167/21214 USt-ID: DE814784953
>>>>> Geschaeftsfuehrer: Mario Scheliga, Rene Otto
>>>>> Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
>>>>> Schoenhauser Allee 51, 10437 Berlin
>>>>>
>>>>
>>>
>>>
>>> --
>>> Sourcegarden GmbH HR: B-104357
>>> Steuernummer: 37/167/21214 USt-ID: DE814784953
>>> Geschaeftsfuehrer: Mario Scheliga, Rene Otto
>>> Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
>>> Schoenhauser Allee 51, 10437 Berlin
>>>
>>
>
>
> --
> Sourcegarden GmbH HR: B-104357
> Steuernummer: 37/167/21214 USt-ID: DE814784953
> Geschaeftsfuehrer: Mario Scheliga, Rene Otto
> Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
> Schoenhauser Allee 51, 10437 Berlin
>
Re: Chaining of views/MapReduce
Posted by Mario Scheliga <ma...@sourcegarden.de>.
Hi Norman,
i found a post by P.Davis that contains a possibility
http://mail-archives.apache.org/mod_mbox/couchdb-user/200910.mbox/%3Ce2111bbb0910280610q37de08ack316c628a4a48cbcc@mail.gmail.com%3E
chaining could be done by some serverside logic, where the result is
stored in a seperate db.
what do you think?
greetz
mario
Am 18.02.2010 um 11:20 schrieb Norman Rosner:
> Hi Mario,
>
> you're probably right about the map phase and read only and all
> that. Hovercraft doesn't look right to me though. I found another
> thread on the mailinglist (http://markmail.org/thread/
> ne2ghnwpojxkhalj) that handles a similar topic I guess.
>
> In Hadoop you can chain jobs (map/reduce phases). So you can take
> the output of the first job as the input of the second job and so
> on. But Hadoop is based on a distributed filesystem, so the results
> are merged together into one location after a job is done, so you
> don't have to think about the thousand servers ;)
>
> I'll guess I will write a workaround in Java to pipe in all the rows
> of the view, extract my stuff, cache it somehow and write it back as
> a new document to the database. After that I could check my rows of
> the view against the newly created doc.
>
> Cheers,
> norman
> On 18.02.2010, at 09:01, Mario Scheliga wrote:
>
>> Hi Norman,
>>
>> i think its obvious that this wont be possible with couchdb itself.
>> but i think hovercraft by jchris can do that for you.
>>
>> http://github.com/jchris/hovercraft
>>
>> otherwise you have to implement the second check after re-reduce on
>> the client side.
>> because, map-function only read data and create new ones (in the
>> map only).
>> writing docs is left to the other processes (put docs or update).
>>
>> could you explain it for me, how you do this in hadoop?
>>
>> greetz
>> mario
>>
>> Am 17.02.2010 um 23:29 schrieb Norman Rosner:
>>
>>>
>>> On 17.02.2010, at 23:15, Mario Scheliga wrote:
>>>
>>>> Hi Norman,
>>>>
>>>> updating a document from map-function its not possible and seems
>>>> to be the wrong way.
>>>> Thinking of map function processing docs seperatly (sandbox), so
>>>> you are able to
>>>> spread the execution over thousand of servers ;-)
>>>
>>> True that! But: suppose I'm just creating/updating one document
>>> per couchdb-instance, that should be ok, right? Because after
>>> that, I can easily get all the result documents and merge them
>>> together. I would do it in as similar way in Hadoop. And as far as
>>> I read in the loooong archives of this list, I'm not the only one
>>> who wants to do such things.
>>>
>>> cheers,
>>> norman
>>>
>>>>
>>>> greez
>>>> mario
>>>>
>>>> Am 17.02.2010 um 21:04 schrieb Norman Rosner:
>>>>
>>>>> Hi folks,
>>>>>
>>>>> first, I'll have to admit that I'm kinda new to JavaScript and
>>>>> of course to CouchDB. Second I just reuse the subject so I hope
>>>>> it also pops up if anybody searches for it.
>>>>>
>>>>> As I read chaining of views is not possible yet but it's
>>>>> mentioned couple of times on the mailing list. So here's what I
>>>>> want to do:
>>>>>
>>>>> 1. Create a list of unique labels/tags/whatever through all of
>>>>> the documents (e.g. all nouns that are in the documents)
>>>>> 2. Extract all labels/tags/nouns of each document and check them
>>>>> again the before calculated result in some kind of way
>>>>>
>>>>> For the second point I created a view which works except of the
>>>>> checking against the result from point 1. Now I'm trying to
>>>>> solve point 1.
>>>>> And here my questions begin: How can I create/update a document
>>>>> from inside of a map function? As I think of it, I'll have to
>>>>> make a HTTP GET to load the document in each iteration. I found
>>>>> some HTTP stuff in the test.js in the test folder, but I'm not
>>>>> quite sure how to use it and if it's the right way of thinking?
>>>>> Is there any way of using global variables throughout
>>>>> 'couchapps' (e.g. through the lib folder and thelike)?
>>>>>
>>>>> Any help of you CouchDB kings would be greatly appreciated!
>>>>>
>>>>> Cheers,
>>>>>
>>>>> norman
>>>>
>>>>
>>>> --
>>>> Sourcegarden GmbH HR: B-104357
>>>> Steuernummer: 37/167/21214 USt-ID: DE814784953
>>>> Geschaeftsfuehrer: Mario Scheliga, Rene Otto
>>>> Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
>>>> Schoenhauser Allee 51, 10437 Berlin
>>>>
>>>
>>
>>
>> --
>> Sourcegarden GmbH HR: B-104357
>> Steuernummer: 37/167/21214 USt-ID: DE814784953
>> Geschaeftsfuehrer: Mario Scheliga, Rene Otto
>> Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
>> Schoenhauser Allee 51, 10437 Berlin
>>
>
--
Sourcegarden GmbH HR: B-104357
Steuernummer: 37/167/21214 USt-ID: DE814784953
Geschaeftsfuehrer: Mario Scheliga, Rene Otto
Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
Schoenhauser Allee 51, 10437 Berlin
Re: Chaining of views/MapReduce
Posted by Norman Rosner <no...@googlemail.com>.
Hi Mario,
you're probably right about the map phase and read only and all that. Hovercraft doesn't look right to me though. I found another thread on the mailinglist (http://markmail.org/thread/ne2ghnwpojxkhalj) that handles a similar topic I guess.
In Hadoop you can chain jobs (map/reduce phases). So you can take the output of the first job as the input of the second job and so on. But Hadoop is based on a distributed filesystem, so the results are merged together into one location after a job is done, so you don't have to think about the thousand servers ;)
I'll guess I will write a workaround in Java to pipe in all the rows of the view, extract my stuff, cache it somehow and write it back as a new document to the database. After that I could check my rows of the view against the newly created doc.
Cheers,
norman
On 18.02.2010, at 09:01, Mario Scheliga wrote:
> Hi Norman,
>
> i think its obvious that this wont be possible with couchdb itself. but i think hovercraft by jchris can do that for you.
>
> http://github.com/jchris/hovercraft
>
> otherwise you have to implement the second check after re-reduce on the client side.
> because, map-function only read data and create new ones (in the map only).
> writing docs is left to the other processes (put docs or update).
>
> could you explain it for me, how you do this in hadoop?
>
> greetz
> mario
>
> Am 17.02.2010 um 23:29 schrieb Norman Rosner:
>
>>
>> On 17.02.2010, at 23:15, Mario Scheliga wrote:
>>
>>> Hi Norman,
>>>
>>> updating a document from map-function its not possible and seems to be the wrong way.
>>> Thinking of map function processing docs seperatly (sandbox), so you are able to
>>> spread the execution over thousand of servers ;-)
>>
>> True that! But: suppose I'm just creating/updating one document per couchdb-instance, that should be ok, right? Because after that, I can easily get all the result documents and merge them together. I would do it in as similar way in Hadoop. And as far as I read in the loooong archives of this list, I'm not the only one who wants to do such things.
>>
>> cheers,
>> norman
>>
>>>
>>> greez
>>> mario
>>>
>>> Am 17.02.2010 um 21:04 schrieb Norman Rosner:
>>>
>>>> Hi folks,
>>>>
>>>> first, I'll have to admit that I'm kinda new to JavaScript and of course to CouchDB. Second I just reuse the subject so I hope it also pops up if anybody searches for it.
>>>>
>>>> As I read chaining of views is not possible yet but it's mentioned couple of times on the mailing list. So here's what I want to do:
>>>>
>>>> 1. Create a list of unique labels/tags/whatever through all of the documents (e.g. all nouns that are in the documents)
>>>> 2. Extract all labels/tags/nouns of each document and check them again the before calculated result in some kind of way
>>>>
>>>> For the second point I created a view which works except of the checking against the result from point 1. Now I'm trying to solve point 1.
>>>> And here my questions begin: How can I create/update a document from inside of a map function? As I think of it, I'll have to make a HTTP GET to load the document in each iteration. I found some HTTP stuff in the test.js in the test folder, but I'm not quite sure how to use it and if it's the right way of thinking? Is there any way of using global variables throughout 'couchapps' (e.g. through the lib folder and thelike)?
>>>>
>>>> Any help of you CouchDB kings would be greatly appreciated!
>>>>
>>>> Cheers,
>>>>
>>>> norman
>>>
>>>
>>> --
>>> Sourcegarden GmbH HR: B-104357
>>> Steuernummer: 37/167/21214 USt-ID: DE814784953
>>> Geschaeftsfuehrer: Mario Scheliga, Rene Otto
>>> Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
>>> Schoenhauser Allee 51, 10437 Berlin
>>>
>>
>
>
> --
> Sourcegarden GmbH HR: B-104357
> Steuernummer: 37/167/21214 USt-ID: DE814784953
> Geschaeftsfuehrer: Mario Scheliga, Rene Otto
> Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
> Schoenhauser Allee 51, 10437 Berlin
>
Re: Chaining of views/MapReduce
Posted by Mario Scheliga <ma...@sourcegarden.de>.
Hi Norman,
i think its obvious that this wont be possible with couchdb itself.
but i think hovercraft by jchris can do that for you.
http://github.com/jchris/hovercraft
otherwise you have to implement the second check after re-reduce on
the client side.
because, map-function only read data and create new ones (in the map
only).
writing docs is left to the other processes (put docs or update).
could you explain it for me, how you do this in hadoop?
greetz
mario
Am 17.02.2010 um 23:29 schrieb Norman Rosner:
>
> On 17.02.2010, at 23:15, Mario Scheliga wrote:
>
>> Hi Norman,
>>
>> updating a document from map-function its not possible and seems to
>> be the wrong way.
>> Thinking of map function processing docs seperatly (sandbox), so
>> you are able to
>> spread the execution over thousand of servers ;-)
>
> True that! But: suppose I'm just creating/updating one document per
> couchdb-instance, that should be ok, right? Because after that, I
> can easily get all the result documents and merge them together. I
> would do it in as similar way in Hadoop. And as far as I read in the
> loooong archives of this list, I'm not the only one who wants to do
> such things.
>
> cheers,
> norman
>
>>
>> greez
>> mario
>>
>> Am 17.02.2010 um 21:04 schrieb Norman Rosner:
>>
>>> Hi folks,
>>>
>>> first, I'll have to admit that I'm kinda new to JavaScript and of
>>> course to CouchDB. Second I just reuse the subject so I hope it
>>> also pops up if anybody searches for it.
>>>
>>> As I read chaining of views is not possible yet but it's mentioned
>>> couple of times on the mailing list. So here's what I want to do:
>>>
>>> 1. Create a list of unique labels/tags/whatever through all of the
>>> documents (e.g. all nouns that are in the documents)
>>> 2. Extract all labels/tags/nouns of each document and check them
>>> again the before calculated result in some kind of way
>>>
>>> For the second point I created a view which works except of the
>>> checking against the result from point 1. Now I'm trying to solve
>>> point 1.
>>> And here my questions begin: How can I create/update a document
>>> from inside of a map function? As I think of it, I'll have to make
>>> a HTTP GET to load the document in each iteration. I found some
>>> HTTP stuff in the test.js in the test folder, but I'm not quite
>>> sure how to use it and if it's the right way of thinking? Is there
>>> any way of using global variables throughout 'couchapps' (e.g.
>>> through the lib folder and thelike)?
>>>
>>> Any help of you CouchDB kings would be greatly appreciated!
>>>
>>> Cheers,
>>>
>>> norman
>>
>>
>> --
>> Sourcegarden GmbH HR: B-104357
>> Steuernummer: 37/167/21214 USt-ID: DE814784953
>> Geschaeftsfuehrer: Mario Scheliga, Rene Otto
>> Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
>> Schoenhauser Allee 51, 10437 Berlin
>>
>
--
Sourcegarden GmbH HR: B-104357
Steuernummer: 37/167/21214 USt-ID: DE814784953
Geschaeftsfuehrer: Mario Scheliga, Rene Otto
Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
Schoenhauser Allee 51, 10437 Berlin
Re: Chaining of views/MapReduce
Posted by Mario Scheliga <ma...@sourcegarden.de>.
but i thought, map-function only read data and create new ones (in the
map only).
writing docs is left to the other processes (put docs or update)
Am 17.02.2010 um 23:29 schrieb Norman Rosner:
>
> On 17.02.2010, at 23:15, Mario Scheliga wrote:
>
>> Hi Norman,
>>
>> updating a document from map-function its not possible and seems to
>> be the wrong way.
>> Thinking of map function processing docs seperatly (sandbox), so
>> you are able to
>> spread the execution over thousand of servers ;-)
>
> True that! But: suppose I'm just creating/updating one document per
> couchdb-instance, that should be ok, right? Because after that, I
> can easily get all the result documents and merge them together. I
> would do it in as similar way in Hadoop. And as far as I read in the
> loooong archives of this list, I'm not the only one who wants to do
> such things.
>
> cheers,
> norman
>
>>
>> greez
>> mario
>>
>> Am 17.02.2010 um 21:04 schrieb Norman Rosner:
>>
>>> Hi folks,
>>>
>>> first, I'll have to admit that I'm kinda new to JavaScript and of
>>> course to CouchDB. Second I just reuse the subject so I hope it
>>> also pops up if anybody searches for it.
>>>
>>> As I read chaining of views is not possible yet but it's mentioned
>>> couple of times on the mailing list. So here's what I want to do:
>>>
>>> 1. Create a list of unique labels/tags/whatever through all of the
>>> documents (e.g. all nouns that are in the documents)
>>> 2. Extract all labels/tags/nouns of each document and check them
>>> again the before calculated result in some kind of way
>>>
>>> For the second point I created a view which works except of the
>>> checking against the result from point 1. Now I'm trying to solve
>>> point 1.
>>> And here my questions begin: How can I create/update a document
>>> from inside of a map function? As I think of it, I'll have to make
>>> a HTTP GET to load the document in each iteration. I found some
>>> HTTP stuff in the test.js in the test folder, but I'm not quite
>>> sure how to use it and if it's the right way of thinking? Is there
>>> any way of using global variables throughout 'couchapps' (e.g.
>>> through the lib folder and thelike)?
>>>
>>> Any help of you CouchDB kings would be greatly appreciated!
>>>
>>> Cheers,
>>>
>>> norman
>>
>>
>> --
>> Sourcegarden GmbH HR: B-104357
>> Steuernummer: 37/167/21214 USt-ID: DE814784953
>> Geschaeftsfuehrer: Mario Scheliga, Rene Otto
>> Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
>> Schoenhauser Allee 51, 10437 Berlin
>>
>
--
Sourcegarden GmbH HR: B-104357
Steuernummer: 37/167/21214 USt-ID: DE814784953
Geschaeftsfuehrer: Mario Scheliga, Rene Otto
Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
Schoenhauser Allee 51, 10437 Berlin
Re: Chaining of views/MapReduce
Posted by Norman Rosner <no...@googlemail.com>.
Am 19.02.2010 um 16:10 schrieb J Chris Anderson <jc...@couch.io>:
>
> On Feb 17, 2010, at 5:29 PM, Norman Rosner wrote:
>
>>
>> On 17.02.2010, at 23:15, Mario Scheliga wrote:
>>
>>> Hi Norman,
>>>
>>> updating a document from map-function its not possible and seems
>>> to be the wrong way.
>>> Thinking of map function processing docs seperatly (sandbox), so
>>> you are able to
>>> spread the execution over thousand of servers ;-)
>>
>> True that! But: suppose I'm just creating/updating one document per
>> couchdb-instance, that should be ok, right? Because after that, I
>> can easily get all the result documents and merge them together. I
>> would do it in as similar way in Hadoop. And as far as I read in
>> the loooong archives of this list, I'm not the only one who wants
>> to do such things.
>
>
> The "proper" way to do this is to have a simple CouchDB map reduce
> view that is the 1st phase of your chain.
>
> Then query the view with group=true and store the output into an
> empty db (one document per row).
>
> Now you can write another view on top of the derived db to do the
> second phase (sort by value, etc).
>
> Chris
>
Hey Chris
in the end I did in a similar way but I was wondering if I could do it
in the map function. But I'm aware now that it is not possible. As I
wrote in the beginning: Im just getting started with CouchDB. But
thanks for the proper Way explanation.
Cheers,
norman
>>
>> cheers,
>> norman
>>
>>>
>>> greez
>>> mario
>>>
>>> Am 17.02.2010 um 21:04 schrieb Norman Rosner:
>>>
>>>> Hi folks,
>>>>
>>>> first, I'll have to admit that I'm kinda new to JavaScript and of
>>>> course to CouchDB. Second I just reuse the subject so I hope it
>>>> also pops up if anybody searches for it.
>>>>
>>>> As I read chaining of views is not possible yet but it's
>>>> mentioned couple of times on the mailing list. So here's what I
>>>> want to do:
>>>>
>>>> 1. Create a list of unique labels/tags/whatever through all of
>>>> the documents (e.g. all nouns that are in the documents)
>>>> 2. Extract all labels/tags/nouns of each document and check them
>>>> again the before calculated result in some kind of way
>>>>
>>>> For the second point I created a view which works except of the
>>>> checking against the result from point 1. Now I'm trying to solve
>>>> point 1.
>>>> And here my questions begin: How can I create/update a document
>>>> from inside of a map function? As I think of it, I'll have to
>>>> make a HTTP GET to load the document in each iteration. I found
>>>> some HTTP stuff in the test.js in the test folder, but I'm not
>>>> quite sure how to use it and if it's the right way of thinking?
>>>> Is there any way of using global variables throughout
>>>> 'couchapps' (e.g. through the lib folder and thelike)?
>>>>
>>>> Any help of you CouchDB kings would be greatly appreciated!
>>>>
>>>> Cheers,
>>>>
>>>> norman
>>>
>>>
>>> --
>>> Sourcegarden GmbH HR: B-104357
>>> Steuernummer: 37/167/21214 USt-ID: DE814784953
>>> Geschaeftsfuehrer: Mario Scheliga, Rene Otto
>>> Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
>>> Schoenhauser Allee 51, 10437 Berlin
>>>
>>
>
Re: Chaining of views/MapReduce
Posted by Simon Metson <si...@googlemail.com>.
Hi,
On 22 Feb 2010, at 18:16, James Marca wrote:
> I see two problems with the current proper way. First, it seems
> wasteful of disk space to have a view generated and then store
> essentially the same thing as a separate db. Second and more
> importantly, as a developer you have to write long-lasting code that
> pays attention to the source database to update the chain of view-
> >db->view->db...->view when the source db data changes. It would be
> nicer if CouchDB could manage all that internally. Perhaps the map
> code could explicitly dump to a db, maybe something like
> emit_chained with a required target db as a third argument, so that
> changes to the source database can get cascaded automatically.
I have some python code that watches for N changes and pulls a view
into a new database. I only knocked it up over the weekend and it uses
a bunch of code from work (which others are welcome to use, but its
currently not that portable) but if you're interested in that I can
point you to it. With the _changes feed it's pretty simple to do,
though I've not yet decided on how the view should propagate over -
there are a lot of ways to deal with that...
Cheers
Simon
Re: Chaining of views/MapReduce
Posted by James Marca <jm...@translab.its.uci.edu>.
On Mon, Feb 22, 2010 at 10:16:23AM -0800, James Marca wrote:
> On Fri, Feb 19, 2010 at 10:10:23AM -0500, J Chris Anderson wrote:
> >
> > On Feb 17, 2010, at 5:29 PM, Norman Rosner wrote:
> >
> > >
> > > On 17.02.2010, at 23:15, Mario Scheliga wrote:
> > >
> > >> Hi Norman,
> > >>
> > >> updating a document from map-function its not possible and seems to be the wrong way.
> > >> Thinking of map function processing docs seperatly (sandbox), so you are able to
> > >> spread the execution over thousand of servers ;-)
> > >
> > > True that! But: suppose I'm just creating/updating one document per couchdb-instance, that should be ok, right? Because after that, I can easily get all the result documents and merge them together. I would do it in as similar way in Hadoop. And as far as I read in the loooong archives of this list, I'm not the only one who wants to do such things.
> >
> >
> > The "proper" way to do this is to have a simple CouchDB map reduce view that is the 1st phase of your chain.
> >
> > Then query the view with group=true and store the output into an empty db (one document per row).
> >
> > Now you can write another view on top of the derived db to do the second phase (sort by value, etc).
>
> Forgive me in advance, I have no erlang skills and no time or ability
> to submit a patch, but I have to ask. Are there any plans in the
> development roadmap to make this less a kludge and more a core
> feature?
Okay, double apologies, I just saw the longer thread hashing out this
topic.
Back to lurking silently.
James
>
> I see two problems with the current proper way. First, it seems
> wasteful of disk space to have a view generated and then store
> essentially the same thing as a separate db. Second and more
> importantly, as a developer you have to write long-lasting code that
> pays attention to the source database to update the chain of
> view->db->view->db...->view when the source db data changes. It would
> be nicer if CouchDB could manage all that internally. Perhaps the map
> code could explicitly dump to a db, maybe something like emit_chained
> with a required target db as a third argument, so that changes to the
> source database can get cascaded automatically.
>
> Regards,
> James
>
> --
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
Re: Chaining of views/MapReduce
Posted by James Marca <jm...@translab.its.uci.edu>.
On Fri, Feb 19, 2010 at 10:10:23AM -0500, J Chris Anderson wrote:
>
> On Feb 17, 2010, at 5:29 PM, Norman Rosner wrote:
>
> >
> > On 17.02.2010, at 23:15, Mario Scheliga wrote:
> >
> >> Hi Norman,
> >>
> >> updating a document from map-function its not possible and seems to be the wrong way.
> >> Thinking of map function processing docs seperatly (sandbox), so you are able to
> >> spread the execution over thousand of servers ;-)
> >
> > True that! But: suppose I'm just creating/updating one document per couchdb-instance, that should be ok, right? Because after that, I can easily get all the result documents and merge them together. I would do it in as similar way in Hadoop. And as far as I read in the loooong archives of this list, I'm not the only one who wants to do such things.
>
>
> The "proper" way to do this is to have a simple CouchDB map reduce view that is the 1st phase of your chain.
>
> Then query the view with group=true and store the output into an empty db (one document per row).
>
> Now you can write another view on top of the derived db to do the second phase (sort by value, etc).
Forgive me in advance, I have no erlang skills and no time or ability
to submit a patch, but I have to ask. Are there any plans in the
development roadmap to make this less a kludge and more a core
feature?
I see two problems with the current proper way. First, it seems
wasteful of disk space to have a view generated and then store
essentially the same thing as a separate db. Second and more
importantly, as a developer you have to write long-lasting code that
pays attention to the source database to update the chain of
view->db->view->db...->view when the source db data changes. It would
be nicer if CouchDB could manage all that internally. Perhaps the map
code could explicitly dump to a db, maybe something like emit_chained
with a required target db as a third argument, so that changes to the
source database can get cascaded automatically.
Regards,
James
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
Re: Chaining of views/MapReduce
Posted by J Chris Anderson <jc...@couch.io>.
On Feb 17, 2010, at 5:29 PM, Norman Rosner wrote:
>
> On 17.02.2010, at 23:15, Mario Scheliga wrote:
>
>> Hi Norman,
>>
>> updating a document from map-function its not possible and seems to be the wrong way.
>> Thinking of map function processing docs seperatly (sandbox), so you are able to
>> spread the execution over thousand of servers ;-)
>
> True that! But: suppose I'm just creating/updating one document per couchdb-instance, that should be ok, right? Because after that, I can easily get all the result documents and merge them together. I would do it in as similar way in Hadoop. And as far as I read in the loooong archives of this list, I'm not the only one who wants to do such things.
The "proper" way to do this is to have a simple CouchDB map reduce view that is the 1st phase of your chain.
Then query the view with group=true and store the output into an empty db (one document per row).
Now you can write another view on top of the derived db to do the second phase (sort by value, etc).
Chris
>
> cheers,
> norman
>
>>
>> greez
>> mario
>>
>> Am 17.02.2010 um 21:04 schrieb Norman Rosner:
>>
>>> Hi folks,
>>>
>>> first, I'll have to admit that I'm kinda new to JavaScript and of course to CouchDB. Second I just reuse the subject so I hope it also pops up if anybody searches for it.
>>>
>>> As I read chaining of views is not possible yet but it's mentioned couple of times on the mailing list. So here's what I want to do:
>>>
>>> 1. Create a list of unique labels/tags/whatever through all of the documents (e.g. all nouns that are in the documents)
>>> 2. Extract all labels/tags/nouns of each document and check them again the before calculated result in some kind of way
>>>
>>> For the second point I created a view which works except of the checking against the result from point 1. Now I'm trying to solve point 1.
>>> And here my questions begin: How can I create/update a document from inside of a map function? As I think of it, I'll have to make a HTTP GET to load the document in each iteration. I found some HTTP stuff in the test.js in the test folder, but I'm not quite sure how to use it and if it's the right way of thinking? Is there any way of using global variables throughout 'couchapps' (e.g. through the lib folder and thelike)?
>>>
>>> Any help of you CouchDB kings would be greatly appreciated!
>>>
>>> Cheers,
>>>
>>> norman
>>
>>
>> --
>> Sourcegarden GmbH HR: B-104357
>> Steuernummer: 37/167/21214 USt-ID: DE814784953
>> Geschaeftsfuehrer: Mario Scheliga, Rene Otto
>> Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
>> Schoenhauser Allee 51, 10437 Berlin
>>
>
Re: Chaining of views/MapReduce
Posted by Norman Rosner <no...@googlemail.com>.
On 17.02.2010, at 23:15, Mario Scheliga wrote:
> Hi Norman,
>
> updating a document from map-function its not possible and seems to be the wrong way.
> Thinking of map function processing docs seperatly (sandbox), so you are able to
> spread the execution over thousand of servers ;-)
True that! But: suppose I'm just creating/updating one document per couchdb-instance, that should be ok, right? Because after that, I can easily get all the result documents and merge them together. I would do it in as similar way in Hadoop. And as far as I read in the loooong archives of this list, I'm not the only one who wants to do such things.
cheers,
norman
>
> greez
> mario
>
> Am 17.02.2010 um 21:04 schrieb Norman Rosner:
>
>> Hi folks,
>>
>> first, I'll have to admit that I'm kinda new to JavaScript and of course to CouchDB. Second I just reuse the subject so I hope it also pops up if anybody searches for it.
>>
>> As I read chaining of views is not possible yet but it's mentioned couple of times on the mailing list. So here's what I want to do:
>>
>> 1. Create a list of unique labels/tags/whatever through all of the documents (e.g. all nouns that are in the documents)
>> 2. Extract all labels/tags/nouns of each document and check them again the before calculated result in some kind of way
>>
>> For the second point I created a view which works except of the checking against the result from point 1. Now I'm trying to solve point 1.
>> And here my questions begin: How can I create/update a document from inside of a map function? As I think of it, I'll have to make a HTTP GET to load the document in each iteration. I found some HTTP stuff in the test.js in the test folder, but I'm not quite sure how to use it and if it's the right way of thinking? Is there any way of using global variables throughout 'couchapps' (e.g. through the lib folder and thelike)?
>>
>> Any help of you CouchDB kings would be greatly appreciated!
>>
>> Cheers,
>>
>> norman
>
>
> --
> Sourcegarden GmbH HR: B-104357
> Steuernummer: 37/167/21214 USt-ID: DE814784953
> Geschaeftsfuehrer: Mario Scheliga, Rene Otto
> Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
> Schoenhauser Allee 51, 10437 Berlin
>
Re: Chaining of views/MapReduce
Posted by Mario Scheliga <ma...@sourcegarden.de>.
Hi Norman,
updating a document from map-function its not possible and seems to be
the wrong way.
Thinking of map function processing docs seperatly (sandbox), so you
are able to
spread the execution over thousand of servers ;-)
greez
mario
Am 17.02.2010 um 21:04 schrieb Norman Rosner:
> Hi folks,
>
> first, I'll have to admit that I'm kinda new to JavaScript and of
> course to CouchDB. Second I just reuse the subject so I hope it
> also pops up if anybody searches for it.
>
> As I read chaining of views is not possible yet but it's mentioned
> couple of times on the mailing list. So here's what I want to do:
>
> 1. Create a list of unique labels/tags/whatever through all of the
> documents (e.g. all nouns that are in the documents)
> 2. Extract all labels/tags/nouns of each document and check them
> again the before calculated result in some kind of way
>
> For the second point I created a view which works except of the
> checking against the result from point 1. Now I'm trying to solve
> point 1.
> And here my questions begin: How can I create/update a document from
> inside of a map function? As I think of it, I'll have to make a HTTP
> GET to load the document in each iteration. I found some HTTP stuff
> in the test.js in the test folder, but I'm not quite sure how to use
> it and if it's the right way of thinking? Is there any way of using
> global variables throughout 'couchapps' (e.g. through the lib folder
> and thelike)?
>
> Any help of you CouchDB kings would be greatly appreciated!
>
> Cheers,
>
> norman
--
Sourcegarden GmbH HR: B-104357
Steuernummer: 37/167/21214 USt-ID: DE814784953
Geschaeftsfuehrer: Mario Scheliga, Rene Otto
Bank: Deutsche Bank, BLZ: 10070024, KTO: 0810929
Schoenhauser Allee 51, 10437 Berlin