You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Noel Quintos <np...@gmail.com> on 2017/05/02 20:50:24 UTC
Return web request via GET with a specific format
Hi!
We have a 3rd party client who expect values to be returned (after a GET
request) in this format:
*[*
* {*
"attributes": *{*
"intensity": *-12*,
"description": "Ground to cloud.",
"strike_time": "2017/04/17 02:14:32.000",
"age": *449*,
"latitude": *46.167*,
"longitude": *-66.81*
* }*
* }*,
* {*
"attributes": *{*
"intensity": *26*,
"description": "Cloud to ground.",
"strike_time": "2017/04/17 02:14:30.000",
"age": *449*,
"latitude": *46.035*,
"longitude": *-66.322*
* },*
*...*
]
However, the values that you would normally get from CouchDB, even with
views, would be something like:
{
"total_rows":2,
"offset":0,
rows: [
{
"id":"223345223355"
"key":'432523235"
"value":
{
"rev":"1-a34234234"
}
"doc":
{
"_id":"73234234"
"_rev":"2-a1324234"
"intensity": *-12*,
"description": "Ground to cloud.",
"strike_time": "2017/04/17 02:14:32.000",
"age": *449*,
"latitude": *46.167*,
"longitude": *-66.81*
* }*
},
{
"id":"123345223355"
"key":'132523235"
"value":
{
"rev":"1-234234234"
}
"doc":
{
"_id":"23234234"
"_rev":"1-a1324234"
"intensity": *26*,
"description": "Cloud to ground.",
"strike_time": "2017/04/17 02:14:30.000",
"age": *449*,
"latitude": *46.035*,
"longitude": *-66.322*
* }*
},
}
It does not matter that "attributes" was replaced with "doc" - that is
fine. The problem has more to do with the number of levels (it will now be
3 levels like xyz["rows"]["doc"]["intensity"] instead of the expected 2
levels like xyz["doc"]["intensity"]; and also the reply format (preferred
is list of objects but couchDB is giving just one object and the list is
just a member.
Is there a way to control the format of the returned value? I have seen
suggestion to use show function but have not seen enough example for this
particular use-case.
Thanks!
--
Noel P. Quintos
Re: Return web request via GET with a specific format
Posted by Noel Quintos <np...@gmail.com>.
Sorry, correction in Step 1. Second to the last line should read:
light = Lightning(information=dict(attributes=dict(timestamp =
datetime.now(), longitude = 30.4, latitude = 18.9, description='cloud to
ground')))
On Mon, May 8, 2017 at 10:07 AM, Noel Quintos <np...@gmail.com> wrote:
> I had a hard time installing couchapp (my machine is not connected to the
> internet) but finally got that working, and I would say it is worth the
> effort. My problem is solved and here is what I did, just in case somebody
> out there have the same problem at hand.
>
> 1. I used python couchdb module to save into couchDb like this:
>
> from couchdb.mapping import Mapping, DictField, Document, TextField,
> FloatField, DateTimeField
> class Lightning(Document):
> information = DictField(Mapping.build(
> attributes = DictField(Mapping.build(
> timestamp = DateTimeField();
> longitude = FloatField(),
> latitude = FloatField(),
> description = TextField()
> ))
> ))
> light = Lightning(timestamp = datetime.now(), longitude = 30.4, latitude =
> 18.9, description='cloud to ground')
> light.store(db)
>
> 2. Using couchapp, I did:
> couchapp init vaisala
> Take note that 'vaisala' is my design document name.
> There might be something wrong with my couchapp installation because it
> did not create the various sub-directories and so, I manually created the
> "lists" directory
>
> 3. I then did:
> couchapp generate list lightlist
> It then created lightlist.js inside the "lists" directory, which I then
> added this code:
>
> function(head, req) {
> var row = getRow();
> var mylist = new Array();
> while(row) {
> mylist.push(row.value.information);
> row=getRow();
> }
> send(JSON.stringify(mylist, null, 2);
> }
>
> Prior to that, I have an existing view called "nv" which is contained on
> another design document called "doc". Notice that in my list function, I am
> simply building a regular javascript array, and the elements are already in
> the right object structure. Recall that my target JSON output is an array
> (hence, I am building an array in my list function) of dictionary of
> dictionaries. "row.value.information" is actually dictionary of
> dictionaries that I see in my "nv" view. At the end of my list function, I
> am simply converting the javascript object to JSON format. The "2" in the
> argument will provide indentation to the JSON output, making it more
> readable, and make it look like more my desired output.
>
> 4. I uploaded the list function via "couchapp push lightning".
> "lightning" here is the name of my couchDB database. Using Fauxton, I am
> able to see in the "lightning" database, an entry with id of
> "_design/vaisala", and under that, "lists", I can see "lightlist"
> containing the function that I just created.
>
> 5. I then type:
> curl -X GET http://172.16.54.8:5984/lightning/_design/vaisala/_
> list/lightlist/doc/nv
> And that showed that data in the format that I want it.
>
> Thanks to everyone who helped me figure it out especially to Aurélien, who
> had been very patient with me!
>
> On Thu, May 4, 2017 at 11:23 PM, Aurélien Bénel <au...@utt.fr>
> wrote:
>
>> Hi Noel,
>>
>> > "error":"compilation_error","reason":"Expression does not eval to a
>> function.
>>
>>
>> We could analyse the JSON manually, but as your application will get
>> bigger,
>> it will become more and more complex.
>> As I wrote earlier, I recommend you to use a tool like couchapp (
>> http://github.com/couchapp/couchapp).
>>
>> The idea is to convert a folder with subfolders and files into a design
>> document.
>> In your case it would be something like:
>>
>> - doc / (weird name for a design document…)
>> - views
>> - map
>> - nv.js
>> - lists
>> - lightning.js
>>
>> Documentation is there:
>> https://couchapp.readthedocs.io/en/latest/couchapp/gettingst
>> arted.html
>>
>> Note that, with couchapp, you are not obliged to generate a complete «
>> app », but maybe it’s easier to begin with.
>>
>>
>> Regards,
>>
>> Aurélien
>
>
>
>
> --
> Noel P. Quintos
>
--
Noel P. Quintos
Re: Return web request via GET with a specific format
Posted by Noel Quintos <np...@gmail.com>.
I had a hard time installing couchapp (my machine is not connected to the
internet) but finally got that working, and I would say it is worth the
effort. My problem is solved and here is what I did, just in case somebody
out there have the same problem at hand.
1. I used python couchdb module to save into couchDb like this:
from couchdb.mapping import Mapping, DictField, Document, TextField,
FloatField, DateTimeField
class Lightning(Document):
information = DictField(Mapping.build(
attributes = DictField(Mapping.build(
timestamp = DateTimeField();
longitude = FloatField(),
latitude = FloatField(),
description = TextField()
))
))
light = Lightning(timestamp = datetime.now(), longitude = 30.4, latitude =
18.9, description='cloud to ground')
light.store(db)
2. Using couchapp, I did:
couchapp init vaisala
Take note that 'vaisala' is my design document name.
There might be something wrong with my couchapp installation because it did
not create the various sub-directories and so, I manually created the
"lists" directory
3. I then did:
couchapp generate list lightlist
It then created lightlist.js inside the "lists" directory, which I then
added this code:
function(head, req) {
var row = getRow();
var mylist = new Array();
while(row) {
mylist.push(row.value.information);
row=getRow();
}
send(JSON.stringify(mylist, null, 2);
}
Prior to that, I have an existing view called "nv" which is contained on
another design document called "doc". Notice that in my list function, I am
simply building a regular javascript array, and the elements are already in
the right object structure. Recall that my target JSON output is an array
(hence, I am building an array in my list function) of dictionary of
dictionaries. "row.value.information" is actually dictionary of
dictionaries that I see in my "nv" view. At the end of my list function, I
am simply converting the javascript object to JSON format. The "2" in the
argument will provide indentation to the JSON output, making it more
readable, and make it look like more my desired output.
4. I uploaded the list function via "couchapp push lightning". "lightning"
here is the name of my couchDB database. Using Fauxton, I am able to see in
the "lightning" database, an entry with id of "_design/vaisala", and under
that, "lists", I can see "lightlist" containing the function that I just
created.
5. I then type:
curl -X GET
http://172.16.54.8:5984/lightning/_design/vaisala/_list/lightlist/doc/nv
And that showed that data in the format that I want it.
Thanks to everyone who helped me figure it out especially to Aurélien, who
had been very patient with me!
On Thu, May 4, 2017 at 11:23 PM, Aurélien Bénel <au...@utt.fr>
wrote:
> Hi Noel,
>
> > "error":"compilation_error","reason":"Expression does not eval to a
> function.
>
>
> We could analyse the JSON manually, but as your application will get
> bigger,
> it will become more and more complex.
> As I wrote earlier, I recommend you to use a tool like couchapp (
> http://github.com/couchapp/couchapp).
>
> The idea is to convert a folder with subfolders and files into a design
> document.
> In your case it would be something like:
>
> - doc / (weird name for a design document…)
> - views
> - map
> - nv.js
> - lists
> - lightning.js
>
> Documentation is there:
> https://couchapp.readthedocs.io/en/latest/couchapp/gettingstarted.html
>
> Note that, with couchapp, you are not obliged to generate a complete « app
> », but maybe it’s easier to begin with.
>
>
> Regards,
>
> Aurélien
--
Noel P. Quintos
Re: Return web request via GET with a specific format
Posted by Aurélien Bénel <au...@utt.fr>.
Hi Noel,
> "error":"compilation_error","reason":"Expression does not eval to a function.
We could analyse the JSON manually, but as your application will get bigger,
it will become more and more complex.
As I wrote earlier, I recommend you to use a tool like couchapp (http://github.com/couchapp/couchapp).
The idea is to convert a folder with subfolders and files into a design document.
In your case it would be something like:
- doc / (weird name for a design document…)
- views
- map
- nv.js
- lists
- lightning.js
Documentation is there:
https://couchapp.readthedocs.io/en/latest/couchapp/gettingstarted.html
Note that, with couchapp, you are not obliged to generate a complete « app », but maybe it’s easier to begin with.
Regards,
Aurélien
Re: Return web request via GET with a specific format
Posted by Noel Quintos <np...@gmail.com>.
I managed to save a list function as shown below in the last row:
$ curl http://172.16.54.8:5984/lightning/_all_docs\?include_docs\=true
{"total_rows":4,"offset":0,"rows":[
{"id":"94fdb18c0294e666f94cea2f53014830","key":"94fdb18c0294e666f94cea2f53014830","value":{"rev":"8-06e54ccb86e59676d5926caebdab7b33"},"doc":{"_id":"94fdb18c0294e666f94cea2f53014830","_rev":"8-06e54ccb86e59676d5926caebdab7b33","still_active":true,"multiplicity":1,"longitude":110.179,"elapsed_time":0,"intensity":0.0,"strike_type":"cloud
to cloud","latitude":58.914,"time_stamp":"2017-05-04T16:23:48.228000Z"}},
{"id":"94fdb18c0294e666f94cea2f5301573f","key":"94fdb18c0294e666f94cea2f5301573f","value":{"rev":"8-e8c840686355d470479a7bc03d8b56e2"},"doc":{"_id":"94fdb18c0294e666f94cea2f5301573f","_rev":"8-e8c840686355d470479a7bc03d8b56e2","still_active":true,"multiplicity":8,"longitude":118.763,"elapsed_time":0,"intensity":0.0,"strike_type":"cloud
to cloud","latitude":58.836,"time_stamp":"2017-05-04T16:23:48.228000Z"}},
{"id":"94fdb18c0294e666f94cea2f53015cfe","key":"94fdb18c0294e666f94cea2f53015cfe","value":{"rev":"8-f4c81acec498a8e2059170de0c477ff9"},"doc":{"_id":"94fdb18c0294e666f94cea2f53015cfe","_rev":"8-f4c81acec498a8e2059170de0c477ff9","still_active":true,"multiplicity":1,"longitude":113.242,"elapsed_time":0,"intensity":0.0,"strike_type":"cloud
to cloud","latitude":52.438,"time_stamp":"2017-05-04T16:23:48.230000Z"}},
{"id":"_design/doc","key":"_design/doc","value":{"rev":"14-4689f641dbc934b9bbc6f0ca8eb4dc2e"},"doc":{"_id":"_design/doc","_rev":"14-4689f641dbc934b9bbc6f0ca8eb4dc2e","id":"_design/doc","views":{"nv":{"map":"function(doc)
{\n emit(null, doc);\n}"}},"lists":{"lightning":"function(head, req) { var
row = getRow(); var firsttime = 1; if (!row) { send('[]'); } send('[\n');
while (row=getRow()) { if (firsttime) { send('\t{\n'); } else {
send('\t,{\n'); firsttime = 0; } send('\t\tattributes:'+row.value);
send('\t}\n'); } send(']\n'); }"}}}
]}
I have a view called "nv" with returned values below:
$ curl http://172.16.54.8:5984/lightning/_design/doc/_view/nv
{"total_rows":3,"offset":0,"rows":[
{"id":"94fdb18c0294e666f94cea2f53014830","key":null,"value":{"_id":"94fdb18c0294e666f94cea2f53014830","_rev":"17-94f163875b0573e3ac3feec283ef78a3","still_active":true,"multiplicity":1,"longitude":110.179,"elapsed_time":1,"intensity":0,"strike_type":"cloud
to cloud","latitude":58.914,"time_stamp":"2017-05-04T16:23:48.228000Z"}},
{"id":"94fdb18c0294e666f94cea2f5301573f","key":null,"value":{"_id":"94fdb18c0294e666f94cea2f5301573f","_rev":"17-81edddee8a3bab95b9b64f0b03c33a47","still_active":true,"multiplicity":8,"longitude":118.763,"elapsed_time":1,"intensity":0,"strike_type":"cloud
to cloud","latitude":58.836,"time_stamp":"2017-05-04T16:23:48.228000Z"}},
{"id":"94fdb18c0294e666f94cea2f53015cfe","key":null,"value":{"_id":"94fdb18c0294e666f94cea2f53015cfe","_rev":"17-ac5a7ead3eedc2e226133064dbd3fda3","still_active":true,"multiplicity":1,"longitude":113.242,"elapsed_time":1,"intensity":0,"strike_type":"cloud
to cloud","latitude":52.438,"time_stamp":"2017-05-04T16:23:48.230000Z"}}
]}
However, if I try viewing that list (apply lists function "lightning" to
view "nv"), I get this error message
$ curl http://172.16.54.8:5984/lightning/_design/doc/_list/lightning/nv
{"error":"compilation_error","reason":"Expression does not eval to a
function. (function(head, req) { var row = getRow(); var firsttime = 1; if
(!row) { send('[]'); } send('[\n'); while (row=getRow()) { if (firsttime) {
send('\t{\n'); } else { send('\t,{\n'); firsttime = 0; }
send('\t\tattributes:'+row.value); send('\t}\n'); } send(']\n'); })"}
I have no javascript background but nevertheless, I believe that is a valid
javascript function and the error might have to do with the JSON data
structure that I made for the lists. Does lists need to have functions
mapped to "map" just like views? I tried doing that the first time but I am
getting error saying:
{"error":"invalid_design_doc","reason":"`lists.lightning` field must have
string type"}
Thanks!
On Wed, May 3, 2017 at 9:34 AM, Aurélien Bénel <au...@utt.fr>
wrote:
> > Formatting a view is done with a *`list`* function.
> > http://docs.couchdb.org/en/latest/couchapp/ddocs.html#list-functions
> >> Yes, it looks like "list" is what I am looking for. I am very new with
> CouchDB and just using Fauxton. I don’t see there any method of creating
> list under Design Document (it does allow me to create views, however). How
> do you create this manually, via Fauxton, if possible? Thanks!
>
> You could create them by editing the design document JSON directly, but it
> would a pain.
> Instead I recommend using command line commands to convert a tree of files
> and folders into a design document.
>
> There are several of them. Here is one I use:
>
> https://github.com/couchapp/couchapp
>
>
> Regards,
>
> Aurélien
--
Noel P. Quintos
Re: Return web request via GET with a specific format
Posted by Noel Quintos <np...@gmail.com>.
Thanks Aurelien! I will take a look and try it out.
On Wed, May 3, 2017 at 9:34 AM, Aurélien Bénel <au...@utt.fr>
wrote:
> > Formatting a view is done with a *`list`* function.
> > http://docs.couchdb.org/en/latest/couchapp/ddocs.html#list-functions
> >> Yes, it looks like "list" is what I am looking for. I am very new with
> CouchDB and just using Fauxton. I don’t see there any method of creating
> list under Design Document (it does allow me to create views, however). How
> do you create this manually, via Fauxton, if possible? Thanks!
>
> You could create them by editing the design document JSON directly, but it
> would a pain.
> Instead I recommend using command line commands to convert a tree of files
> and folders into a design document.
>
> There are several of them. Here is one I use:
>
> https://github.com/couchapp/couchapp
>
>
> Regards,
>
> Aurélien
--
Noel P. Quintos
Re: Return web request via GET with a specific format
Posted by Aurélien Bénel <au...@utt.fr>.
> Formatting a view is done with a *`list`* function.
> http://docs.couchdb.org/en/latest/couchapp/ddocs.html#list-functions
>> Yes, it looks like "list" is what I am looking for. I am very new with CouchDB and just using Fauxton. I don’t see there any method of creating list under Design Document (it does allow me to create views, however). How do you create this manually, via Fauxton, if possible? Thanks!
You could create them by editing the design document JSON directly, but it would a pain.
Instead I recommend using command line commands to convert a tree of files and folders into a design document.
There are several of them. Here is one I use:
https://github.com/couchapp/couchapp
Regards,
Aurélien
Re: Return web request via GET with a specific format
Posted by Noel Quintos <np...@gmail.com>.
Hi Aurelien!
Yes, it looks like "list" is what I am looking for. I am very new with
CouchDB and just using Fauxton. I don't see there any method of creating
list under Design Document (it does allow me to create views, however). How
do you create this manually, via Fauxton, if possible? Thanks!
On Tue, May 2, 2017 at 10:59 PM, Aurélien Bénel <au...@utt.fr>
wrote:
> Hi Noel,
>
> >> However, the values that you would normally get from CouchDB, even with
> >> views, would be something like: (…)
> > This site explains show functions: http://guide.couchdb.org/
> draft/show.html
> > You would just output it in JSON, rather than HTML like the example
> shows.
>
> Not exactly: `show` functions are just to format a *single* document.
> Formatting a view is done with a *`list`* function.
>
> http://docs.couchdb.org/en/latest/couchapp/ddocs.html#list-functions
>
>
> Regards,
>
> Aurélien
--
Noel P. Quintos
Re: Return web request via GET with a specific format
Posted by Aurélien Bénel <au...@utt.fr>.
Hi Noel,
>> However, the values that you would normally get from CouchDB, even with
>> views, would be something like: (…)
> This site explains show functions: http://guide.couchdb.org/draft/show.html
> You would just output it in JSON, rather than HTML like the example shows.
Not exactly: `show` functions are just to format a *single* document.
Formatting a view is done with a *`list`* function.
http://docs.couchdb.org/en/latest/couchapp/ddocs.html#list-functions
Regards,
Aurélien
Re: Return web request via GET with a specific format
Posted by Noel Quintos <np...@gmail.com>.
Thanks Sean! However, it appears that "show" would format only one document
per request and would not be applied for all documents in the database,
unlike view.
On Tue, May 2, 2017 at 8:58 PM, Sean Lang <sl...@gmail.com> wrote:
> This site explains show functions: http://guide.couchdb.org/
> draft/show.html
>
> You would just output it in JSON, rather than HTML like the example shows.
>
> On Tue, May 2, 2017, 15:50 Noel Quintos <np...@gmail.com> wrote:
>
> > Hi!
> >
> > We have a 3rd party client who expect values to be returned (after a GET
> > request) in this format:
> >
> > *[*
> > * {*
> > "attributes": *{*
> > "intensity": *-12*,
> > "description": "Ground to cloud.",
> > "strike_time": "2017/04/17 02:14:32.000",
> > "age": *449*,
> > "latitude": *46.167*,
> > "longitude": *-66.81*
> > * }*
> > * }*,
> > * {*
> > "attributes": *{*
> > "intensity": *26*,
> > "description": "Cloud to ground.",
> > "strike_time": "2017/04/17 02:14:30.000",
> > "age": *449*,
> > "latitude": *46.035*,
> > "longitude": *-66.322*
> > * },*
> > *...*
> > ]
> > However, the values that you would normally get from CouchDB, even with
> > views, would be something like:
> >
> > {
> > "total_rows":2,
> > "offset":0,
> > rows: [
> > {
> > "id":"223345223355"
> > "key":'432523235"
> > "value":
> > {
> > "rev":"1-a34234234"
> > }
> > "doc":
> > {
> > "_id":"73234234"
> > "_rev":"2-a1324234"
> > "intensity": *-12*,
> > "description": "Ground to cloud.",
> > "strike_time": "2017/04/17 02:14:32.000",
> > "age": *449*,
> > "latitude": *46.167*,
> > "longitude": *-66.81*
> > * }*
> > },
> > {
> > "id":"123345223355"
> > "key":'132523235"
> > "value":
> > {
> > "rev":"1-234234234"
> > }
> > "doc":
> > {
> > "_id":"23234234"
> > "_rev":"1-a1324234"
> > "intensity": *26*,
> > "description": "Cloud to ground.",
> > "strike_time": "2017/04/17 02:14:30.000",
> > "age": *449*,
> > "latitude": *46.035*,
> > "longitude": *-66.322*
> > * }*
> > },
> > }
> >
> > It does not matter that "attributes" was replaced with "doc" - that is
> > fine. The problem has more to do with the number of levels (it will now
> be
> > 3 levels like xyz["rows"]["doc"]["intensity"] instead of the expected 2
> > levels like xyz["doc"]["intensity"]; and also the reply format (preferred
> > is list of objects but couchDB is giving just one object and the list is
> > just a member.
> >
> > Is there a way to control the format of the returned value? I have seen
> > suggestion to use show function but have not seen enough example for this
> > particular use-case.
> >
> > Thanks!
> >
> >
> > --
> > Noel P. Quintos
> >
>
--
Noel P. Quintos
Re: Return web request via GET with a specific format
Posted by Sean Lang <sl...@gmail.com>.
This site explains show functions: http://guide.couchdb.org/draft/show.html
You would just output it in JSON, rather than HTML like the example shows.
On Tue, May 2, 2017, 15:50 Noel Quintos <np...@gmail.com> wrote:
> Hi!
>
> We have a 3rd party client who expect values to be returned (after a GET
> request) in this format:
>
> *[*
> * {*
> "attributes": *{*
> "intensity": *-12*,
> "description": "Ground to cloud.",
> "strike_time": "2017/04/17 02:14:32.000",
> "age": *449*,
> "latitude": *46.167*,
> "longitude": *-66.81*
> * }*
> * }*,
> * {*
> "attributes": *{*
> "intensity": *26*,
> "description": "Cloud to ground.",
> "strike_time": "2017/04/17 02:14:30.000",
> "age": *449*,
> "latitude": *46.035*,
> "longitude": *-66.322*
> * },*
> *...*
> ]
> However, the values that you would normally get from CouchDB, even with
> views, would be something like:
>
> {
> "total_rows":2,
> "offset":0,
> rows: [
> {
> "id":"223345223355"
> "key":'432523235"
> "value":
> {
> "rev":"1-a34234234"
> }
> "doc":
> {
> "_id":"73234234"
> "_rev":"2-a1324234"
> "intensity": *-12*,
> "description": "Ground to cloud.",
> "strike_time": "2017/04/17 02:14:32.000",
> "age": *449*,
> "latitude": *46.167*,
> "longitude": *-66.81*
> * }*
> },
> {
> "id":"123345223355"
> "key":'132523235"
> "value":
> {
> "rev":"1-234234234"
> }
> "doc":
> {
> "_id":"23234234"
> "_rev":"1-a1324234"
> "intensity": *26*,
> "description": "Cloud to ground.",
> "strike_time": "2017/04/17 02:14:30.000",
> "age": *449*,
> "latitude": *46.035*,
> "longitude": *-66.322*
> * }*
> },
> }
>
> It does not matter that "attributes" was replaced with "doc" - that is
> fine. The problem has more to do with the number of levels (it will now be
> 3 levels like xyz["rows"]["doc"]["intensity"] instead of the expected 2
> levels like xyz["doc"]["intensity"]; and also the reply format (preferred
> is list of objects but couchDB is giving just one object and the list is
> just a member.
>
> Is there a way to control the format of the returned value? I have seen
> suggestion to use show function but have not seen enough example for this
> particular use-case.
>
> Thanks!
>
>
> --
> Noel P. Quintos
>