You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by GitBox <gi...@apache.org> on 2019/12/26 22:53:14 UTC
[GitHub] [couchdb] akircher opened a new issue #2383: OS Process Error
akircher opened a new issue #2383: OS Process Error
URL: https://github.com/apache/couchdb/issues/2383
[NOTE]: #
A several views will not compile on a fresh install of CouchDB 2.3.1. They fail with the following:
http: `{"error":"os_process_error","reason":"{exit_status,1}","ref":4089041121}`
logs:
```
[info] 2019-12-26T22:29:21.028998Z couchdb@127.0.0.1 <0.227.0> -------- couch_proc_manager <0.23626.187> died normal
[error] 2019-12-26T22:29:21.029229Z couchdb@127.0.0.1 <0.4416.235> -------- OS Process Error <0.23626.187> :: {os_process_error,{exit_status,1}}
[error] 2019-12-26T22:29:21.030147Z couchdb@127.0.0.1 emulator -------- Error in process <0.4416.235> on node 'couchdb@127.0.0.1' with exit value:
{{nocatch,{os_process_error,{exit_status,1}}},[{couch_os_process,prompt,2,[{file,"src/couch_os_process.erl"},{line,59}]},{couch_query_servers,map_doc_raw,2,[{file,$
[error] 2019-12-26T22:29:21.031315Z couchdb@127.0.0.1 <0.9675.235> 769c1b9dd2 rexi_server: from: couchdb@127.0.0.1(<0.9537.235>) mfa: fabric_rpc:reduce_view/4 thro$
[error] 2019-12-26T22:29:21.031683Z couchdb@127.0.0.1 <0.9537.235> 769c1b9dd2 req_err(4089041121) os_process_error : {exit_status,1}
[<<"couch_mrview_util:get_view_index_state/5 L137">>,<<"couch_mrview_util:get_view/4 L81">>,<<"couch_mrview:query_view/6 L247">>,<<"rexi_server:init_p/3 L140">$
```
## Description
I have tried increasing
`[couchdb] os_process_timeout to 120000`; and
`[fabric] request_timeout to 120000`
I have also tried deleting the database and re-replicating it but none of the above have worked. Documents are all small and look like this:
```
{
"_id": "2017-06-21T19:19:26.979555Z",
"_rev": "22-f78246e507328744138d77328ae67a87",
"qty": {
"from": null,
"to": 3
},
"exp": {
"from": null,
"to": "2017-09-30T04:00:00.000Z"
},
"drug": {
"_id": "0378-6232",
"price": {
"goodrx": 0.2333,
"nadac": 0.0324,
"updatedAt": "2016-11-15T21:55:03.794Z"
},
"brand": "Celexa",
"generic": "Citalopram 20mg",
"generics": [
{
"name": "Citalopram",
"strength": "20mg"
}
],
"form": "Tablet",
"gsns": "46203"
},
"user": {
"_id": "8889875187"
},
"shipment": {
"_id": "8889875187.2017-06-21T19:14:48.7275249333"
},
"bin": null,
"next": [
{
"disposed": {},
"createdAt": "2017-06-21T19:19:26.979Z"
}
],
"updatedAt": "2019-12-16T19:21:19.333Z",
"verifiedAt": null
}
```
Can someone let me know if this is a known bug (related to https://github.com/apache/couchdb/issues/1966?) or if I am doing something wrong.
## Your Environment
AWS EC2 Ubuntu 16.04LTS
CouchDB:
```
echo "deb https://apache.bintray.com/couchdb-deb xenial main" | sudo tee -a /etc/apt/sources.list #note: xenial is for Ubuntu 16
curl -L https://couchdb.apache.org/repo/bintray-pubkey.asc | sudo apt-key add -
sudo apt-get update && sudo apt-get install couchdb
```
Thank you in advance for the help and your work on such a great project!!!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [couchdb] akircher commented on issue #2383: OS Process Error
Posted by GitBox <gi...@apache.org>.
akircher commented on issue #2383: OS Process Error
URL: https://github.com/apache/couchdb/issues/2383#issuecomment-569447496
This appears to be an error within the view and not with CouchDB. I had a `for loop` in the code that would iterate (and `emit`) several times per document. However I had a few (~100) misformed documents that caused the `for loop` to run emit about a 1000 times on each of the 100 malformed documents. Once I put a check into the view to make sure the loop only ran on well-formed documents the views began to compile.
In an ideal world, CouchDB would have a better error message here to help quickly diagnose the issue but I know the team is busy with v3.0 and v4.0 so I think that is a better use of their time than improving this error message. Thus I am closing for now, but feel free to reopen if someone wants to work on this.
Thanks again to Nick for the help. You all make the couchdb community great and welcoming.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [couchdb] akircher commented on issue #2383: OS Process Error
Posted by GitBox <gi...@apache.org>.
akircher commented on issue #2383: OS Process Error
URL: https://github.com/apache/couchdb/issues/2383#issuecomment-569189380
Hi Nick,
Thanks for your quick and thoughtful response!
I am using the built-in `_stats_ reducer but am return an array of 2 values so I get 2 stats objects.
I modified the `opts/couchdb/bin/couchdb` file as you suggested trying both 32MB and 64MB (67108864) values but unfortunately the same error.
Any other ideas of what could be happening?
Thanks
Adam
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [couchdb] akircher edited a comment on issue #2383: OS Process Error
Posted by GitBox <gi...@apache.org>.
akircher edited a comment on issue #2383: OS Process Error
URL: https://github.com/apache/couchdb/issues/2383#issuecomment-569189380
Hi Nick,
Thanks for your quick and thoughtful response!
I am using the built-in `_stats` reducer but am emitting an array of 2 `values` for each `key` so I get 2 stats objects. I modified the `opts/couchdb/bin/couchdb` file as you suggested trying both 32MB and 64MB (67108864) values and restarted couchdb but unfortunately the same error.
Any other ideas of what could be happening?
Thanks
Adam
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [couchdb] nickva commented on issue #2383: OS Process Error
Posted by GitBox <gi...@apache.org>.
nickva commented on issue #2383: OS Process Error
URL: https://github.com/apache/couchdb/issues/2383#issuecomment-569159160
@akircher thanks for your report.
Do you by chance use a custom reduce?
(Asking since saw the ` fabric_rpc:reduce_view/4` line the traceback above)..
If you do, make sure the reduce function reduces the values it outputs, otherwise you could be running out of memory or get time outs.
If your reduce function is ok, you might consider increasing the default couchjs stack size. Try setting to say 32MB and see if it makes a difference.
In 2.3.* it's specified as a OS environment variable only not via the *.ini files. I could modify it by editing the bin/couchdb file
```
...
export COUCHDB_QUERY_SERVER_JAVASCRIPT="./bin/couchjs ./share/server/main.js -S 33554430"
...
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [couchdb] akircher edited a comment on issue #2383: OS Process Error
Posted by GitBox <gi...@apache.org>.
akircher edited a comment on issue #2383: OS Process Error
URL: https://github.com/apache/couchdb/issues/2383#issuecomment-569189380
Hi Nick,
Thanks for your quick and thoughtful response!
I am using the built-in `_stats` reducer but am return an array of 2 values so I get 2 stats objects.
I modified the `opts/couchdb/bin/couchdb` file as you suggested trying both 32MB and 64MB (67108864) values but unfortunately the same error.
Any other ideas of what could be happening?
Thanks
Adam
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [couchdb] akircher closed issue #2383: OS Process Error
Posted by GitBox <gi...@apache.org>.
akircher closed issue #2383: OS Process Error
URL: https://github.com/apache/couchdb/issues/2383
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [couchdb] akircher edited a comment on issue #2383: OS Process Error
Posted by GitBox <gi...@apache.org>.
akircher edited a comment on issue #2383: OS Process Error
URL: https://github.com/apache/couchdb/issues/2383#issuecomment-569189380
Hi Nick,
Thanks for your quick and thoughtful response!
I am using the built-in `_stats` reducer but am emitting an array of 2 `values` for each `key` so I get 2 stats objects. And I am emitting those 5-10x times per document (so its a [very?] large view). I modified the `opts/couchdb/bin/couchdb` file as you suggested trying both 32MB and 64MB (67108864) values and restarted couchdb but unfortunately the same error.
Any other ideas of what could be happening?
Thanks
Adam
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [couchdb] nickva commented on issue #2383: OS Process Error
Posted by GitBox <gi...@apache.org>.
nickva commented on issue #2383: OS Process Error
URL: https://github.com/apache/couchdb/issues/2383#issuecomment-569388484
Hi Adam,
Thanks for giving that setting a try. I had now realized that 64Mb was actually the default, wonder if 128Mb would make any difference, then? Maybe there is a way to monitor memory usage of the machine to see if there were any unexpected large spikes around when the crash was reported.
Another thing that could be happening is that the values emitted are not conforming to what a `_stats` reducer would expect. https://docs.couchdb.org/en/stable/ddocs/ddocs.html#_stats. There are some rules about those:
> _stats can operate on key-value pairs where each value is an array comprised of numbers or pre-aggregated objects. In this case every value emitted from the map function must be an array, and the arrays must all be the same length
Perhaps there is a way to confirm that the shape of emitted values matches those rules.
Another idea is to replace the stats reducer with a count or sum reducer and see if it would exhibit the crash as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services