You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by David Alan Hjelle <da...@thehjellejar.com> on 2018/05/24 21:27:52 UTC
`allocation size overflow` when building map-reduce view on 2.1.1
cluster
I’m getting an `allocation size overflow` error [1] when building a map-reduce view on a 2.1.1 3-node cluster on CentOS [2]. The view code works for many other databases on the same cluster, and works on an identical database on CouchDB 1.6.1 [3]. The database isn’t particularly large, nor should it have particularly large documents. I’ve already tried deleting it on the cluster and re-replicating it from 1.6.1 but still had the same issue.
Any pointers on how I can get this to work or what the issue is?
[1] Some more from the log is:
```
[error] 2018-05-23T13:50:43.694720Z couchdb@couch1.iconcmo.com <0.2456.927> -------- OS Process Error <0.30745.549> :: {<<"InternalError">>,{[{<<"message">>,<<"allocation size overflow">>},{<<"fileName">>,<<"./share/server/main.js">>},{<<"lineNumber">>,1435},{<<"stack">>,<<"([object Object])@./share/server/main.js:1435\n()@./share/server/main.js:1582\n()@./share/server/main.js:1603\n@./share/server/main.js:1\n">>}]}}
[error] 2018-05-23T13:50:43.695289Z couchdb@couch1.iconcmo.com <0.17488.921> c9dcebec5c rexi_server: from: couchdb@couch1.iconcmo.com(<0.22850.927>) mfa: fabric_rpc:map_view/5 throw:{<<"InternalError">>,{[{<<"message">>,<<"allocation size overflow">>},{<<"fileName">>,<<"./share/server/main.js">>},{<<"lineNumber">>,1435},{<<"stack">>,<<"([object Object])@./share/server/main.js:1435\n()@./share/server/main.js:1582\n()@./share/server/main.js:1603\n@./share/server/main.js:1\n">>}]}} [{couch_mrview_util,get_view_index_state,5,[{file,"src/couch_mrview_util.erl"},{line,101}]},{couch_mrview_util,get_view,4,[{file,"src/couch_mrview_util.erl"},{line,45}]},{couch_mrview,query_view,6,[{file,"src/couch_mrview.erl"},{line,244}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,139}]}]
[error] 2018-05-23T13:50:43.695471Z couchdb@couch1.iconcmo.com <0.12444.927> fb72b09346 rexi_server: from: couchdb@couch3.iconcmo.com(<13951.16552.873>) mfa: fabric_rpc:map_view/5 throw:{<<"InternalError">>,{[{<<"message">>,<<"allocation size overflow">>},{<<"fileName">>,<<"./share/server/main.js">>},{<<"lineNumber">>,1435},{<<"stack">>,<<"([object Object])@./share/server/main.js:1435\n()@./share/server/main.js:1582\n()@./share/server/main.js:1603\n@./share/server/main.js:1\n">>}]}} [{couch_mrview_util,get_view_index_state,5,[{file,"src/couch_mrview_util.erl"},{line,101}]},{couch_mrview_util,get_view,4,[{file,"src/couch_mrview_util.erl"},{line,45}]},{couch_mrview,query_view,6,[{file,"src/couch_mrview.erl"},{line,244}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,139}]}]
[error] 2018-05-23T13:50:43.696337Z couchdb@couch1.iconcmo.com emulator -------- Error in process <0.2456.927> on node 'couchdb@couch1.iconcmo.com' with exit value:
{{nocatch,{<<"InternalError">>,{[{<<"message">>,<<"allocation size overflow">>},{<<"fileName">>,<<"./share/server/main.js">>},{<<"lineNumber">>,1435},{<<"stack">>,<<"([object Object])@./share/server/main.js:1435\n()@./share/server/main.js:1582\n()@./share/server/main.js:1603\n@./share/server/main.js:1\n">>}]}}},[{couch_os_process,prompt,2,[{file,"src/couch_os_process.erl"},{line,59}]},{couch_query_servers,map_doc_raw,2,[{file,"src/couch_query_servers.erl"},{line,67}]},{couch_mrview_updater,'-map_docs/2-fun-0-',3,[{file,"src/couch_mrview_updater.erl"},{line,195}]},{lists,foldl,3,[{file,"lists.erl"},{line,1262}]},{couch_mrview_updater,map_docs,2,[{file,"src/couch_mrview_updater.erl"},{line,202}]}]}
```
[2] I did downgrade the js package to the working version.
[3] I replicated the data from 1.6.1 originally.
David Alan Hjelle
1 Corinthians 2:2
http://thehjellejar.com/ <http://thehjellejar.com/>
<http://thehjellejar.com/>See the church management software I build at Icon Systems, Inc. <http://iconcmo.com/>
Check out Rita’s spoons <http://jarofwood.com/>.
Re: `allocation size overflow` when building map-reduce view on 2.1.1
cluster
Posted by David Alan Hjelle <da...@thehjellejar.com>.
Thank you for the insight!
I can’t share, unfortunately, but you did lead me to finding a document that was much bigger than I realized. (I put a log(doc._id) into the view, and was eventually able to track down the document that was the cause…) I see I need to adjust our views to handle this more appropriately.
In the meantime, is there a configuration setting anywhere that can influence this? I looked through default.ini, but I didn’t see anything obvious…
David Alan Hjelle
> On May 24, 2018, at 5:05 PM, Joan Touzet <wohali@apache.org <ma...@apache.org>> wrote:
>
> The most likely answer is that your emit()s across all views within a single design document are, collectively, too large.
>
> Would you be so kind as to share your design documents and a sample document or two, please?
>
> -Joan
>
> ----- Original Message -----
> From: "David Alan Hjelle" <dahjelle+couchdb.org@thehjellejar.com <ma...@thehjellejar.com>>
> To: user@couchdb.apache.org <ma...@couchdb.apache.org>
> Sent: Thursday, May 24, 2018 5:27:52 PM
> Subject: `allocation size overflow` when building map-reduce view on 2.1.1 cluster
>
> I’m getting an `allocation size overflow` error [1] when building a map-reduce view on a 2.1.1 3-node cluster on CentOS [2]. The view code works for many other databases on the same cluster, and works on an identical database on CouchDB 1.6.1 [3]. The database isn’t particularly large, nor should it have particularly large documents. I’ve already tried deleting it on the cluster and re-replicating it from 1.6.1 but still had the same issue.
>
> Any pointers on how I can get this to work or what the issue is?
>
>
>
> [1] Some more from the log is:
>
> ```
> [error] 2018-05-23T13:50:43.694720Z couchdb@couch1.iconcmo.com <ma...@couch1.iconcmo.com> <0.2456.927> -------- OS Process Error <0.30745.549> :: {<<"InternalError">>,{[{<<"message">>,<<"allocation size overflow">>},{<<"fileName">>,<<"./share/server/main.js">>},{<<"lineNumber">>,1435},{<<"stack">>,<<"([object Object])@./share/server/main.js:1435\n()@./share/server/main.js:1582\n()@./share/server/main.js:1603\n@./share/server/main.js:1\n">>}]}}
> [error] 2018-05-23T13:50:43.695289Z couchdb@couch1.iconcmo.com <ma...@couch1.iconcmo.com> <0.17488.921> c9dcebec5c rexi_server: from: couchdb@couch1.iconcmo.com <ma...@couch1.iconcmo.com>(<0.22850.927>) mfa: fabric_rpc:map_view/5 throw:{<<"InternalError">>,{[{<<"message">>,<<"allocation size overflow">>},{<<"fileName">>,<<"./share/server/main.js">>},{<<"lineNumber">>,1435},{<<"stack">>,<<"([object Object])@./share/server/main.js:1435\n()@./share/server/main.js:1582\n()@./share/server/main.js:1603\n@./share/server/main.js:1\n">>}]}} [{couch_mrview_util,get_view_index_state,5,[{file,"src/couch_mrview_util.erl"},{line,101}]},{couch_mrview_util,get_view,4,[{file,"src/couch_mrview_util.erl"},{line,45}]},{couch_mrview,query_view,6,[{file,"src/couch_mrview.erl"},{line,244}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,139}]}]
> [error] 2018-05-23T13:50:43.695471Z couchdb@couch1.iconcmo.com <ma...@couch1.iconcmo.com> <0.12444.927> fb72b09346 rexi_server: from: couchdb@couch3.iconcmo.com <ma...@couch3.iconcmo.com>(<13951.16552.873>) mfa: fabric_rpc:map_view/5 throw:{<<"InternalError">>,{[{<<"message">>,<<"allocation size overflow">>},{<<"fileName">>,<<"./share/server/main.js">>},{<<"lineNumber">>,1435},{<<"stack">>,<<"([object Object])@./share/server/main.js:1435\n()@./share/server/main.js:1582\n()@./share/server/main.js:1603\n@./share/server/main.js:1\n">>}]}} [{couch_mrview_util,get_view_index_state,5,[{file,"src/couch_mrview_util.erl"},{line,101}]},{couch_mrview_util,get_view,4,[{file,"src/couch_mrview_util.erl"},{line,45}]},{couch_mrview,query_view,6,[{file,"src/couch_mrview.erl"},{line,244}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,139}]}]
> [error] 2018-05-23T13:50:43.696337Z couchdb@couch1.iconcmo.com <ma...@couch1.iconcmo.com> emulator -------- Error in process <0.2456.927> on node 'couchdb@couch1.iconcmo.com <ma...@couch1.iconcmo.com>' with exit value:
> {{nocatch,{<<"InternalError">>,{[{<<"message">>,<<"allocation size overflow">>},{<<"fileName">>,<<"./share/server/main.js">>},{<<"lineNumber">>,1435},{<<"stack">>,<<"([object Object])@./share/server/main.js:1435\n()@./share/server/main.js:1582\n()@./share/server/main.js:1603\n@./share/server/main.js:1\n">>}]}}},[{couch_os_process,prompt,2,[{file,"src/couch_os_process.erl"},{line,59}]},{couch_query_servers,map_doc_raw,2,[{file,"src/couch_query_servers.erl"},{line,67}]},{couch_mrview_updater,'-map_docs/2-fun-0-',3,[{file,"src/couch_mrview_updater.erl"},{line,195}]},{lists,foldl,3,[{file,"lists.erl"},{line,1262}]},{couch_mrview_updater,map_docs,2,[{file,"src/couch_mrview_updater.erl"},{line,202}]}]}
> ```
>
> [2] I did downgrade the js package to the working version.
> [3] I replicated the data from 1.6.1 originally.
>
>
> David Alan Hjelle
> 1 Corinthians 2:2
> http://thehjellejar.com/ <http://thehjellejar.com/> <http://thehjellejar.com/ <http://thehjellejar.com/>>
>
> <http://thehjellejar.com/ <http://thehjellejar.com/>>See the church management software I build at Icon Systems, Inc. <http://iconcmo.com/ <http://iconcmo.com/>>
> Check out Rita’s spoons <http://jarofwood.com/ <http://jarofwood.com/>>.
David Alan Hjelle
1 Corinthians 2:2
http://thehjellejar.com/ <http://thehjellejar.com/>
<http://thehjellejar.com/>See the church management software I build at Icon Systems, Inc. <http://iconcmo.com/>
Check out Rita’s spoons <http://jarofwood.com/>.
Re: `allocation size overflow` when building map-reduce view on
2.1.1 cluster
Posted by Joan Touzet <wo...@apache.org>.
I'm afraid not. There is a way to increase the stack size if you're doing a lot of recursion, but an allocation size overflow means you've hit the limit of what the JS subsystem can malloc()...
----- Original Message -----
From: "David Alan Hjelle" <da...@thehjellejar.com>
To: user@couchdb.apache.org, "Joan Touzet" <wo...@apache.org>
Sent: Friday, May 25, 2018 11:31:34 AM
Subject: Re: `allocation size overflow` when building map-reduce view on 2.1.1 cluster
Thank you for the insight!
I can’t share, unfortunately, but you did lead me to finding a document that was much bigger than I realized. (I put a log(doc._id) into the view, and was eventually able to track down the document that was the cause…) I see I need to adjust our views to handle this more appropriately.
In the meantime, is there a configuration setting anywhere that can influence this? I looked through default.ini, but I didn’t see anything obvious…
David Alan Hjelle
On May 24, 2018, at 5:05 PM, Joan Touzet < wohali@apache.org > wrote:
The most likely answer is that your emit()s across all views within a single design document are, collectively, too large.
Would you be so kind as to share your design documents and a sample document or two, please?
-Joan
----- Original Message -----
From: "David Alan Hjelle" < dahjelle+couchdb.org@thehjellejar.com >
To: user@couchdb.apache.org
Sent: Thursday, May 24, 2018 5:27:52 PM
Subject: `allocation size overflow` when building map-reduce view on 2.1.1 cluster
I’m getting an `allocation size overflow` error [1] when building a map-reduce view on a 2.1.1 3-node cluster on CentOS [2]. The view code works for many other databases on the same cluster, and works on an identical database on CouchDB 1.6.1 [3]. The database isn’t particularly large, nor should it have particularly large documents. I’ve already tried deleting it on the cluster and re-replicating it from 1.6.1 but still had the same issue.
Any pointers on how I can get this to work or what the issue is?
[1] Some more from the log is:
```
[error] 2018-05-23T13:50:43.694720Z couchdb@couch1.iconcmo.com <0.2456.927> -------- OS Process Error <0.30745.549> :: {<<"InternalError">>,{[{<<"message">>,<<"allocation size overflow">>},{<<"fileName">>,<<"./share/server/main.js">>},{<<"lineNumber">>,1435},{<<"stack">>,<<"([object Object])@./share/server/main.js:1435\n()@./share/server/main.js:1582\n()@./share/server/main.js:1603\n@./share/server/main.js:1\n">>}]}}
[error] 2018-05-23T13:50:43.695289Z couchdb@couch1.iconcmo.com <0.17488.921> c9dcebec5c rexi_server: from: couchdb@couch1.iconcmo.com (<0.22850.927>) mfa: fabric_rpc:map_view/5 throw:{<<"InternalError">>,{[{<<"message">>,<<"allocation size overflow">>},{<<"fileName">>,<<"./share/server/main.js">>},{<<"lineNumber">>,1435},{<<"stack">>,<<"([object Object])@./share/server/main.js:1435\n()@./share/server/main.js:1582\n()@./share/server/main.js:1603\n@./share/server/main.js:1\n">>}]}} [{couch_mrview_util,get_view_index_state,5,[{file,"src/couch_mrview_util.erl"},{line,101}]},{couch_mrview_util,get_view,4,[{file,"src/couch_mrview_util.erl"},{line,45}]},{couch_mrview,query_view,6,[{file,"src/couch_mrview.erl"},{line,244}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,139}]}]
[error] 2018-05-23T13:50:43.695471Z couchdb@couch1.iconcmo.com <0.12444.927> fb72b09346 rexi_server: from: couchdb@couch3.iconcmo.com (<13951.16552.873>) mfa: fabric_rpc:map_view/5 throw:{<<"InternalError">>,{[{<<"message">>,<<"allocation size overflow">>},{<<"fileName">>,<<"./share/server/main.js">>},{<<"lineNumber">>,1435},{<<"stack">>,<<"([object Object])@./share/server/main.js:1435\n()@./share/server/main.js:1582\n()@./share/server/main.js:1603\n@./share/server/main.js:1\n">>}]}} [{couch_mrview_util,get_view_index_state,5,[{file,"src/couch_mrview_util.erl"},{line,101}]},{couch_mrview_util,get_view,4,[{file,"src/couch_mrview_util.erl"},{line,45}]},{couch_mrview,query_view,6,[{file,"src/couch_mrview.erl"},{line,244}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,139}]}]
[error] 2018-05-23T13:50:43.696337Z couchdb@couch1.iconcmo.com emulator -------- Error in process <0.2456.927> on node ' couchdb@couch1.iconcmo.com ' with exit value:
{{nocatch,{<<"InternalError">>,{[{<<"message">>,<<"allocation size overflow">>},{<<"fileName">>,<<"./share/server/main.js">>},{<<"lineNumber">>,1435},{<<"stack">>,<<"([object Object])@./share/server/main.js:1435\n()@./share/server/main.js:1582\n()@./share/server/main.js:1603\n@./share/server/main.js:1\n">>}]}}},[{couch_os_process,prompt,2,[{file,"src/couch_os_process.erl"},{line,59}]},{couch_query_servers,map_doc_raw,2,[{file,"src/couch_query_servers.erl"},{line,67}]},{couch_mrview_updater,'-map_docs/2-fun-0-',3,[{file,"src/couch_mrview_updater.erl"},{line,195}]},{lists,foldl,3,[{file,"lists.erl"},{line,1262}]},{couch_mrview_updater,map_docs,2,[{file,"src/couch_mrview_updater.erl"},{line,202}]}]}
```
[2] I did downgrade the js package to the working version.
[3] I replicated the data from 1.6.1 originally.
David Alan Hjelle
1 Corinthians 2:2
http://thehjellejar.com/ < http://thehjellejar.com/ >
< http://thehjellejar.com/ >See the church management software I build at Icon Systems, Inc. < http://iconcmo.com/ >
Check out Rita’s spoons < http://jarofwood.com/ >.
Re: `allocation size overflow` when building map-reduce view on
2.1.1 cluster
Posted by Joan Touzet <wo...@apache.org>.
The most likely answer is that your emit()s across all views within a single design document are, collectively, too large.
Would you be so kind as to share your design documents and a sample document or two, please?
-Joan
----- Original Message -----
From: "David Alan Hjelle" <da...@thehjellejar.com>
To: user@couchdb.apache.org
Sent: Thursday, May 24, 2018 5:27:52 PM
Subject: `allocation size overflow` when building map-reduce view on 2.1.1 cluster
I’m getting an `allocation size overflow` error [1] when building a map-reduce view on a 2.1.1 3-node cluster on CentOS [2]. The view code works for many other databases on the same cluster, and works on an identical database on CouchDB 1.6.1 [3]. The database isn’t particularly large, nor should it have particularly large documents. I’ve already tried deleting it on the cluster and re-replicating it from 1.6.1 but still had the same issue.
Any pointers on how I can get this to work or what the issue is?
[1] Some more from the log is:
```
[error] 2018-05-23T13:50:43.694720Z couchdb@couch1.iconcmo.com <0.2456.927> -------- OS Process Error <0.30745.549> :: {<<"InternalError">>,{[{<<"message">>,<<"allocation size overflow">>},{<<"fileName">>,<<"./share/server/main.js">>},{<<"lineNumber">>,1435},{<<"stack">>,<<"([object Object])@./share/server/main.js:1435\n()@./share/server/main.js:1582\n()@./share/server/main.js:1603\n@./share/server/main.js:1\n">>}]}}
[error] 2018-05-23T13:50:43.695289Z couchdb@couch1.iconcmo.com <0.17488.921> c9dcebec5c rexi_server: from: couchdb@couch1.iconcmo.com(<0.22850.927>) mfa: fabric_rpc:map_view/5 throw:{<<"InternalError">>,{[{<<"message">>,<<"allocation size overflow">>},{<<"fileName">>,<<"./share/server/main.js">>},{<<"lineNumber">>,1435},{<<"stack">>,<<"([object Object])@./share/server/main.js:1435\n()@./share/server/main.js:1582\n()@./share/server/main.js:1603\n@./share/server/main.js:1\n">>}]}} [{couch_mrview_util,get_view_index_state,5,[{file,"src/couch_mrview_util.erl"},{line,101}]},{couch_mrview_util,get_view,4,[{file,"src/couch_mrview_util.erl"},{line,45}]},{couch_mrview,query_view,6,[{file,"src/couch_mrview.erl"},{line,244}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,139}]}]
[error] 2018-05-23T13:50:43.695471Z couchdb@couch1.iconcmo.com <0.12444.927> fb72b09346 rexi_server: from: couchdb@couch3.iconcmo.com(<13951.16552.873>) mfa: fabric_rpc:map_view/5 throw:{<<"InternalError">>,{[{<<"message">>,<<"allocation size overflow">>},{<<"fileName">>,<<"./share/server/main.js">>},{<<"lineNumber">>,1435},{<<"stack">>,<<"([object Object])@./share/server/main.js:1435\n()@./share/server/main.js:1582\n()@./share/server/main.js:1603\n@./share/server/main.js:1\n">>}]}} [{couch_mrview_util,get_view_index_state,5,[{file,"src/couch_mrview_util.erl"},{line,101}]},{couch_mrview_util,get_view,4,[{file,"src/couch_mrview_util.erl"},{line,45}]},{couch_mrview,query_view,6,[{file,"src/couch_mrview.erl"},{line,244}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,139}]}]
[error] 2018-05-23T13:50:43.696337Z couchdb@couch1.iconcmo.com emulator -------- Error in process <0.2456.927> on node 'couchdb@couch1.iconcmo.com' with exit value:
{{nocatch,{<<"InternalError">>,{[{<<"message">>,<<"allocation size overflow">>},{<<"fileName">>,<<"./share/server/main.js">>},{<<"lineNumber">>,1435},{<<"stack">>,<<"([object Object])@./share/server/main.js:1435\n()@./share/server/main.js:1582\n()@./share/server/main.js:1603\n@./share/server/main.js:1\n">>}]}}},[{couch_os_process,prompt,2,[{file,"src/couch_os_process.erl"},{line,59}]},{couch_query_servers,map_doc_raw,2,[{file,"src/couch_query_servers.erl"},{line,67}]},{couch_mrview_updater,'-map_docs/2-fun-0-',3,[{file,"src/couch_mrview_updater.erl"},{line,195}]},{lists,foldl,3,[{file,"lists.erl"},{line,1262}]},{couch_mrview_updater,map_docs,2,[{file,"src/couch_mrview_updater.erl"},{line,202}]}]}
```
[2] I did downgrade the js package to the working version.
[3] I replicated the data from 1.6.1 originally.
David Alan Hjelle
1 Corinthians 2:2
http://thehjellejar.com/ <http://thehjellejar.com/>
<http://thehjellejar.com/>See the church management software I build at Icon Systems, Inc. <http://iconcmo.com/>
Check out Rita’s spoons <http://jarofwood.com/>.