You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Noah Slater (JIRA)" <ji...@apache.org> on 2008/03/09 01:01:47 UTC
[jira] Created: (COUCHDB-25) Javascript view on big dataset crashes
couchjs?
Javascript view on big dataset crashes couchjs?
-----------------------------------------------
Key: COUCHDB-25
URL: https://issues.apache.org/jira/browse/COUCHDB-25
Project: CouchDB
Issue Type: Bug
Reporter: Noah Slater
Priority: Minor
What steps will reproduce the problem?
Not entirely sure. I have been doing some testing with CouchDB, essentially
downloading messages from twitter for a few days and dumping them in a
database. I would attach the DB file, but it's like 120MB big. Then when I
run a simple view on it, even the simple:
function(doc) {
map(null, doc);
}
I get to see the following messages in the server log. Sometimes it works
though, so it seems kind of arbitrary:
-----
[error] [emulator] Error in process <0.2776.1> with exit value:
{{nocatch,{map_process_error,{exit_status,0}}},[{couch_query_servers,readline,2},{couch_query_servers,read_json,1},{couch_query_servers,'-map_docs/2-fun-0-',2},{lists,map,2},{lists,map,2},{couch_query_servers,map_docs...
=ERROR REPORT==== 7-Mar-2008::15:45:48 ===
Error in process <0.2776.1> with exit value:
{{nocatch,{map_process_error,{exit_status,0}}},[{couch_query_servers,readline,2},{couch_query_servers,read_json,1},{couch_query_servers,'-map_docs/2-fun-0-',2},{lists,map,2},{lists,map,2},{couch_query_servers,map_docs...
[info] [<0.2775.1>] HTTP Error (code 500): {obj,[{error,"map_process_error"},
{reason,"{exit_status,0}"}]}
[info] [<0.2775.1>] HTTP Response Code:500
-----
What version of the product are you using? On what operating system?
This only occurs on the subversion trunk build on Ubuntu Linux 7.10. On the
current release version it works fine.
Please provide any additional information below.
If you want the .couch files with which this happens, let me know.
--
Okay, can you make the DB file available somehow and send me a link to it to cmlenz at gmx.de? I think it
should compress fairly well BTW.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Work started: (COUCHDB-25) Javascript view on big dataset
crashes couchjs?
Posted by "Christopher Lenz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-25?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on COUCHDB-25 started by Christopher Lenz.
> Javascript view on big dataset crashes couchjs?
> -----------------------------------------------
>
> Key: COUCHDB-25
> URL: https://issues.apache.org/jira/browse/COUCHDB-25
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Reporter: Noah Slater
> Assignee: Christopher Lenz
> Priority: Minor
> Attachments: issue25.diff
>
>
> What steps will reproduce the problem?
> Not entirely sure. I have been doing some testing with CouchDB, essentially
> downloading messages from twitter for a few days and dumping them in a
> database. I would attach the DB file, but it's like 120MB big. Then when I
> run a simple view on it, even the simple:
> function(doc) {
> map(null, doc);
> }
> I get to see the following messages in the server log. Sometimes it works
> though, so it seems kind of arbitrary:
> -----
> [error] [emulator] Error in process <0.2776.1> with exit value:
> {{nocatch,{map_process_error,{exit_status,0}}},[{couch_query_servers,readline,2},{couch_query_servers,read_json,1},{couch_query_servers,'-map_docs/2-fun-0-',2},{lists,map,2},{lists,map,2},{couch_query_servers,map_docs...
> =ERROR REPORT==== 7-Mar-2008::15:45:48 ===
> Error in process <0.2776.1> with exit value:
> {{nocatch,{map_process_error,{exit_status,0}}},[{couch_query_servers,readline,2},{couch_query_servers,read_json,1},{couch_query_servers,'-map_docs/2-fun-0-',2},{lists,map,2},{lists,map,2},{couch_query_servers,map_docs...
> [info] [<0.2775.1>] HTTP Error (code 500): {obj,[{error,"map_process_error"},
> {reason,"{exit_status,0}"}]}
> [info] [<0.2775.1>] HTTP Response Code:500
> -----
> What version of the product are you using? On what operating system?
> This only occurs on the subversion trunk build on Ubuntu Linux 7.10. On the
> current release version it works fine.
> Please provide any additional information below.
>
> If you want the .couch files with which this happens, let me know.
> --
> Okay, can you make the DB file available somehow and send me a link to it to cmlenz at gmx.de? I think it
> should compress fairly well BTW.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-25) Javascript view on big dataset crashes
couchjs?
Posted by "Christopher Lenz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-25?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christopher Lenz updated COUCHDB-25:
------------------------------------
Fix Version/s: 0.8
> Javascript view on big dataset crashes couchjs?
> -----------------------------------------------
>
> Key: COUCHDB-25
> URL: https://issues.apache.org/jira/browse/COUCHDB-25
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Reporter: Noah Slater
> Assignee: Christopher Lenz
> Priority: Minor
> Fix For: 0.8
>
> Attachments: issue25.diff
>
>
> What steps will reproduce the problem?
> Not entirely sure. I have been doing some testing with CouchDB, essentially
> downloading messages from twitter for a few days and dumping them in a
> database. I would attach the DB file, but it's like 120MB big. Then when I
> run a simple view on it, even the simple:
> function(doc) {
> map(null, doc);
> }
> I get to see the following messages in the server log. Sometimes it works
> though, so it seems kind of arbitrary:
> -----
> [error] [emulator] Error in process <0.2776.1> with exit value:
> {{nocatch,{map_process_error,{exit_status,0}}},[{couch_query_servers,readline,2},{couch_query_servers,read_json,1},{couch_query_servers,'-map_docs/2-fun-0-',2},{lists,map,2},{lists,map,2},{couch_query_servers,map_docs...
> =ERROR REPORT==== 7-Mar-2008::15:45:48 ===
> Error in process <0.2776.1> with exit value:
> {{nocatch,{map_process_error,{exit_status,0}}},[{couch_query_servers,readline,2},{couch_query_servers,read_json,1},{couch_query_servers,'-map_docs/2-fun-0-',2},{lists,map,2},{lists,map,2},{couch_query_servers,map_docs...
> [info] [<0.2775.1>] HTTP Error (code 500): {obj,[{error,"map_process_error"},
> {reason,"{exit_status,0}"}]}
> [info] [<0.2775.1>] HTTP Response Code:500
> -----
> What version of the product are you using? On what operating system?
> This only occurs on the subversion trunk build on Ubuntu Linux 7.10. On the
> current release version it works fine.
> Please provide any additional information below.
>
> If you want the .couch files with which this happens, let me know.
> --
> Okay, can you make the DB file available somehow and send me a link to it to cmlenz at gmx.de? I think it
> should compress fairly well BTW.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (COUCHDB-25) Javascript view on big dataset
crashes couchjs?
Posted by "Christopher Lenz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-25?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christopher Lenz reassigned COUCHDB-25:
---------------------------------------
Assignee: Christopher Lenz
> Javascript view on big dataset crashes couchjs?
> -----------------------------------------------
>
> Key: COUCHDB-25
> URL: https://issues.apache.org/jira/browse/COUCHDB-25
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Reporter: Noah Slater
> Assignee: Christopher Lenz
> Priority: Minor
>
> What steps will reproduce the problem?
> Not entirely sure. I have been doing some testing with CouchDB, essentially
> downloading messages from twitter for a few days and dumping them in a
> database. I would attach the DB file, but it's like 120MB big. Then when I
> run a simple view on it, even the simple:
> function(doc) {
> map(null, doc);
> }
> I get to see the following messages in the server log. Sometimes it works
> though, so it seems kind of arbitrary:
> -----
> [error] [emulator] Error in process <0.2776.1> with exit value:
> {{nocatch,{map_process_error,{exit_status,0}}},[{couch_query_servers,readline,2},{couch_query_servers,read_json,1},{couch_query_servers,'-map_docs/2-fun-0-',2},{lists,map,2},{lists,map,2},{couch_query_servers,map_docs...
> =ERROR REPORT==== 7-Mar-2008::15:45:48 ===
> Error in process <0.2776.1> with exit value:
> {{nocatch,{map_process_error,{exit_status,0}}},[{couch_query_servers,readline,2},{couch_query_servers,read_json,1},{couch_query_servers,'-map_docs/2-fun-0-',2},{lists,map,2},{lists,map,2},{couch_query_servers,map_docs...
> [info] [<0.2775.1>] HTTP Error (code 500): {obj,[{error,"map_process_error"},
> {reason,"{exit_status,0}"}]}
> [info] [<0.2775.1>] HTTP Response Code:500
> -----
> What version of the product are you using? On what operating system?
> This only occurs on the subversion trunk build on Ubuntu Linux 7.10. On the
> current release version it works fine.
> Please provide any additional information below.
>
> If you want the .couch files with which this happens, let me know.
> --
> Okay, can you make the DB file available somehow and send me a link to it to cmlenz at gmx.de? I think it
> should compress fairly well BTW.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-25) Javascript view on big dataset
crashes couchjs?
Posted by "Brent Calder (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-25?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12578574#action_12578574 ]
Brent Calder commented on COUCHDB-25:
-------------------------------------
I get the same error on my MacOS X (10.5) build (CouchDB 0.7.3a776). No problem with a small database with a few records, but using a 1.7GB, ~150000 record database the outcome is the same:
[error] [emulator] Error in process <0.121.0> with exit value: {{nocatch,{map_process_error,{exit_status,0}}},[{couch_query_servers,readline,2},{couch_query_servers,read_json,1},{couch_query_servers,'-map_docs/2-fun-0-',2},{lists,map,2},{lists,map,2},{couch_query_servers,map_docs...
=ERROR REPORT==== 13-Mar-2008::19:22:37 ===
Error in process <0.121.0> with exit value: {{nocatch,{map_process_error,{exit_status,0}}},[{couch_query_servers,readline,2},{couch_query_servers,read_json,1},{couch_query_servers,'-map_docs/2-fun-0-',2},{lists,map,2},{lists,map,2},{couch_query_servers,map_docs...
[info] [<0.120.0>] HTTP Error (code 500): {obj,[{error,"map_process_error"},
{reason,"{exit_status,0}"}]}
[info] [<0.120.0>] HTTP Response Code:500
> Javascript view on big dataset crashes couchjs?
> -----------------------------------------------
>
> Key: COUCHDB-25
> URL: https://issues.apache.org/jira/browse/COUCHDB-25
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Reporter: Noah Slater
> Priority: Minor
>
> What steps will reproduce the problem?
> Not entirely sure. I have been doing some testing with CouchDB, essentially
> downloading messages from twitter for a few days and dumping them in a
> database. I would attach the DB file, but it's like 120MB big. Then when I
> run a simple view on it, even the simple:
> function(doc) {
> map(null, doc);
> }
> I get to see the following messages in the server log. Sometimes it works
> though, so it seems kind of arbitrary:
> -----
> [error] [emulator] Error in process <0.2776.1> with exit value:
> {{nocatch,{map_process_error,{exit_status,0}}},[{couch_query_servers,readline,2},{couch_query_servers,read_json,1},{couch_query_servers,'-map_docs/2-fun-0-',2},{lists,map,2},{lists,map,2},{couch_query_servers,map_docs...
> =ERROR REPORT==== 7-Mar-2008::15:45:48 ===
> Error in process <0.2776.1> with exit value:
> {{nocatch,{map_process_error,{exit_status,0}}},[{couch_query_servers,readline,2},{couch_query_servers,read_json,1},{couch_query_servers,'-map_docs/2-fun-0-',2},{lists,map,2},{lists,map,2},{couch_query_servers,map_docs...
> [info] [<0.2775.1>] HTTP Error (code 500): {obj,[{error,"map_process_error"},
> {reason,"{exit_status,0}"}]}
> [info] [<0.2775.1>] HTTP Response Code:500
> -----
> What version of the product are you using? On what operating system?
> This only occurs on the subversion trunk build on Ubuntu Linux 7.10. On the
> current release version it works fine.
> Please provide any additional information below.
>
> If you want the .couch files with which this happens, let me know.
> --
> Okay, can you make the DB file available somehow and send me a link to it to cmlenz at gmx.de? I think it
> should compress fairly well BTW.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (COUCHDB-25) Javascript view on big dataset
crashes couchjs?
Posted by "Christopher Lenz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-25?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christopher Lenz resolved COUCHDB-25.
-------------------------------------
Resolution: Fixed
Patch applied in r642448.
> Javascript view on big dataset crashes couchjs?
> -----------------------------------------------
>
> Key: COUCHDB-25
> URL: https://issues.apache.org/jira/browse/COUCHDB-25
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Reporter: Noah Slater
> Assignee: Christopher Lenz
> Priority: Minor
> Attachments: issue25.diff
>
>
> What steps will reproduce the problem?
> Not entirely sure. I have been doing some testing with CouchDB, essentially
> downloading messages from twitter for a few days and dumping them in a
> database. I would attach the DB file, but it's like 120MB big. Then when I
> run a simple view on it, even the simple:
> function(doc) {
> map(null, doc);
> }
> I get to see the following messages in the server log. Sometimes it works
> though, so it seems kind of arbitrary:
> -----
> [error] [emulator] Error in process <0.2776.1> with exit value:
> {{nocatch,{map_process_error,{exit_status,0}}},[{couch_query_servers,readline,2},{couch_query_servers,read_json,1},{couch_query_servers,'-map_docs/2-fun-0-',2},{lists,map,2},{lists,map,2},{couch_query_servers,map_docs...
> =ERROR REPORT==== 7-Mar-2008::15:45:48 ===
> Error in process <0.2776.1> with exit value:
> {{nocatch,{map_process_error,{exit_status,0}}},[{couch_query_servers,readline,2},{couch_query_servers,read_json,1},{couch_query_servers,'-map_docs/2-fun-0-',2},{lists,map,2},{lists,map,2},{couch_query_servers,map_docs...
> [info] [<0.2775.1>] HTTP Error (code 500): {obj,[{error,"map_process_error"},
> {reason,"{exit_status,0}"}]}
> [info] [<0.2775.1>] HTTP Response Code:500
> -----
> What version of the product are you using? On what operating system?
> This only occurs on the subversion trunk build on Ubuntu Linux 7.10. On the
> current release version it works fine.
> Please provide any additional information below.
>
> If you want the .couch files with which this happens, let me know.
> --
> Okay, can you make the DB file available somehow and send me a link to it to cmlenz at gmx.de? I think it
> should compress fairly well BTW.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-25) Javascript view on big dataset crashes
couchjs?
Posted by "Christopher Lenz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-25?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christopher Lenz updated COUCHDB-25:
------------------------------------
Component/s: JavaScript View Server
> Javascript view on big dataset crashes couchjs?
> -----------------------------------------------
>
> Key: COUCHDB-25
> URL: https://issues.apache.org/jira/browse/COUCHDB-25
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Reporter: Noah Slater
> Priority: Minor
>
> What steps will reproduce the problem?
> Not entirely sure. I have been doing some testing with CouchDB, essentially
> downloading messages from twitter for a few days and dumping them in a
> database. I would attach the DB file, but it's like 120MB big. Then when I
> run a simple view on it, even the simple:
> function(doc) {
> map(null, doc);
> }
> I get to see the following messages in the server log. Sometimes it works
> though, so it seems kind of arbitrary:
> -----
> [error] [emulator] Error in process <0.2776.1> with exit value:
> {{nocatch,{map_process_error,{exit_status,0}}},[{couch_query_servers,readline,2},{couch_query_servers,read_json,1},{couch_query_servers,'-map_docs/2-fun-0-',2},{lists,map,2},{lists,map,2},{couch_query_servers,map_docs...
> =ERROR REPORT==== 7-Mar-2008::15:45:48 ===
> Error in process <0.2776.1> with exit value:
> {{nocatch,{map_process_error,{exit_status,0}}},[{couch_query_servers,readline,2},{couch_query_servers,read_json,1},{couch_query_servers,'-map_docs/2-fun-0-',2},{lists,map,2},{lists,map,2},{couch_query_servers,map_docs...
> [info] [<0.2775.1>] HTTP Error (code 500): {obj,[{error,"map_process_error"},
> {reason,"{exit_status,0}"}]}
> [info] [<0.2775.1>] HTTP Response Code:500
> -----
> What version of the product are you using? On what operating system?
> This only occurs on the subversion trunk build on Ubuntu Linux 7.10. On the
> current release version it works fine.
> Please provide any additional information below.
>
> If you want the .couch files with which this happens, let me know.
> --
> Okay, can you make the DB file available somehow and send me a link to it to cmlenz at gmx.de? I think it
> should compress fairly well BTW.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-25) Javascript view on big dataset crashes
couchjs?
Posted by "Christopher Lenz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-25?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christopher Lenz updated COUCHDB-25:
------------------------------------
Attachment: issue25.diff
The problem is caused by a hard-coded branch limit in our SpiderMonkey wrapper, which was added to combat potential memory leaks. As this issue shows, that was a bad idea, as it breaks creating view indices over large datasets. Doh.
The attached patch removes the branch limit, and also fixes a memory leak in the decoding of strings in the readline() function. Seems to work nicely here.
> Javascript view on big dataset crashes couchjs?
> -----------------------------------------------
>
> Key: COUCHDB-25
> URL: https://issues.apache.org/jira/browse/COUCHDB-25
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Reporter: Noah Slater
> Assignee: Christopher Lenz
> Priority: Minor
> Attachments: issue25.diff
>
>
> What steps will reproduce the problem?
> Not entirely sure. I have been doing some testing with CouchDB, essentially
> downloading messages from twitter for a few days and dumping them in a
> database. I would attach the DB file, but it's like 120MB big. Then when I
> run a simple view on it, even the simple:
> function(doc) {
> map(null, doc);
> }
> I get to see the following messages in the server log. Sometimes it works
> though, so it seems kind of arbitrary:
> -----
> [error] [emulator] Error in process <0.2776.1> with exit value:
> {{nocatch,{map_process_error,{exit_status,0}}},[{couch_query_servers,readline,2},{couch_query_servers,read_json,1},{couch_query_servers,'-map_docs/2-fun-0-',2},{lists,map,2},{lists,map,2},{couch_query_servers,map_docs...
> =ERROR REPORT==== 7-Mar-2008::15:45:48 ===
> Error in process <0.2776.1> with exit value:
> {{nocatch,{map_process_error,{exit_status,0}}},[{couch_query_servers,readline,2},{couch_query_servers,read_json,1},{couch_query_servers,'-map_docs/2-fun-0-',2},{lists,map,2},{lists,map,2},{couch_query_servers,map_docs...
> [info] [<0.2775.1>] HTTP Error (code 500): {obj,[{error,"map_process_error"},
> {reason,"{exit_status,0}"}]}
> [info] [<0.2775.1>] HTTP Response Code:500
> -----
> What version of the product are you using? On what operating system?
> This only occurs on the subversion trunk build on Ubuntu Linux 7.10. On the
> current release version it works fine.
> Please provide any additional information below.
>
> If you want the .couch files with which this happens, let me know.
> --
> Okay, can you make the DB file available somehow and send me a link to it to cmlenz at gmx.de? I think it
> should compress fairly well BTW.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.