You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Paweł Melon (JIRA)" <ji...@apache.org> on 2008/10/31 22:05:44 UTC
[jira] Created: (COUCHDB-149) Compactation failure when *.compact
file reach 2116687115 (ap. 2GB) size
Compactation failure when *.compact file reach 2116687115 (ap. 2GB) size
------------------------------------------------------------------------
Key: COUCHDB-149
URL: https://issues.apache.org/jira/browse/COUCHDB-149
Project: CouchDB
Issue Type: Bug
Components: Database Core
Affects Versions: 0.8.1
Environment: Ubuntu 8.04 2.6.24-21-generic x86_64 GNU/Linux processor Duo 2 Quad, 8 GB RAM
Reporter: Paweł Melon
CouchDB is has been working 3 days without stop. I tried to insert large amount of data. From time to time I was doing compactation.
One when database has 50GB of size and 3 mln records, then tried second time when there was 4,7 mln records and 12 GB but something failed. I tried to restart compaction from futon interface but it wasn't working.
On console where was working inserting script written in erlang I received message.
Err add {event,{session,965,3,2005718},
{session_data,os,"Ubuntu 8.10",icecast,"WinampMPEG/5.53"},
{listener,999999999999999,12301897,"85.112.221.200"},
1213297703,undefined,21,undefined,undefined} {ok,
{{"HTTP/1.1",
500,
"Internal Server Error"},
[{"cache-control",
"must-revalidate"},
{"date",
"Fri, 31 Oct 2008 19:13:43 GMT"},
{"server",
"CouchDB/0.8.1-incubating (Erlang OTP/R12B)"},
{"content-length",
"1665"},
{"content-type",
"text/plain;charset=utf-8"}],
"{\"error\":\"EXIT\",\"reason\":\"{{function_clause,\\n [{couch_db,'-copy_docs\\/3-fun-0-',\\n [not_found,<0.24367.262>,\\n {stream,<0.25264.262>,<0.25263.262>},\\n <0.25263.262>]},\\n {lists,map,2},\\n {lists,map,2},\\n {couch_db,copy_docs,3},\\n {couch_db,'-copy_compact_docs\\/2-fun-0-',4},\\n {couch_btree,stream_kv_node2,7},\\n {couch_btree,stream_kp_node,6},\\n {couch_btree,stream_kp_node,7}]},\\n {gen_server,call,\\n [<0.24368.262>,\\n {update_docs,\\n [[{doc,\\\"965:3:2005718\\\",\\n [\\\"102178913\\\"],\\n {obj,\\n [{\\\"type\\\",\\\"event\\\"},\\n {\\\"session\\\",\\n {obj,\\n [{\\\"server_id\\\",965},\\n {\\\"instance_id\\\",3},\\n {\\\"id\\\",2005718}]}},\\n {\\\"session_data\\\",\\n {obj,\\n [{\\\"origin\\\",\\\"os\\\"},\\n {\\\"os_type\\\",\\\"Ubuntu 8.10\\\"},\\n {\\\"server_type\\\",\\\"icecast\\\"},\\n {\\\"player\\\",\\\"WinampMPEG\\/5.53\\\"}]}},\\n {\\\"listener\\\",\\n {obj,\\n [{\\\"pid\\\",999999999999999},\\n {\\\"gg\\\",12301897},\\n {\\\"ip\\\",\\\"85.112.221.200\\\"}]}},\\n {\\\"start_time\\\",1213297703},\\n {\\\"end_time\\\",\\\"undefined\\\"},\\n {\\\"channel\\\",21},\\n {\\\"intro\\\",\\\"undefined\\\"}]},\\n [],false,[]}]],\\n [new_edits]},\\n infinity]}}\"}"}}
I think that the most important part is error reason:
"{\"error\":\"EXIT\",\"reason\":\"{{function_clause,\\n [{couch_db,'-copy_docs\\/3-fun-0-',\\n [not_found,<0.24367.262>,\\n {stream,<0.25264.262>,<0.25263.262>},\\n <0.25263.262>]},\\n {lists,map,2},\\n {lists,map,2},\\n {couch_db,copy_docs,3},\\n {couch_db,'-copy_compact_docs\\/2-fun-0-',4},\\n {couch_btree,stream_kv_node2,7},\\n {couch_btree,stream_kp_node,6},\\n {couch_btree,stream_kp_node,7}]},\\n {gen_server,call,\\n [<0.24368.262>,\\n {update_docs,\\n [[{doc,\\\"965:3:2005718\\\",\\n [\\\"102178913\\\"],\\n {obj,\\n [{\\\"type\\\",\\\"event\\\"},\\n {\\\"session\\\",\\n {obj,\\n [{\\\"server_id\\\",965},\\n
or second time
"{\"error\":\"EXIT\",\"reason\":\"{noproc,{gen_server,call,\\n [<0.25341.262>,{pread_bin,18695772967},infinity]}}\"}"}
It is strange that compaction alway failed when *.compact file has 2116687115 or 2116963527 bytes of size. Always approximately 2GB.
Is it bug? or there is some config parameter which says about result compact file size. If here is one, why file before compaction can has 50 GB size?
Im sory for my poor English.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (COUCHDB-149) Compactation failure when *.compact
file reach 2116687115 (ap. 2GB) size
Posted by "Damien Katz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Damien Katz resolved COUCHDB-149.
---------------------------------
Resolution: Fixed
Fix Version/s: 0.9
Assignee: Damien Katz
This looks very similar to a bug that is fixed in trunk. If you can reproduce this with a recent trunk, please reopen.
> Compactation failure when *.compact file reach 2116687115 (ap. 2GB) size
> ------------------------------------------------------------------------
>
> Key: COUCHDB-149
> URL: https://issues.apache.org/jira/browse/COUCHDB-149
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Affects Versions: 0.8.1
> Environment: Ubuntu 8.04 2.6.24-21-generic x86_64 GNU/Linux processor Duo 2 Quad, 8 GB RAM
> Reporter: Paweł Melon
> Assignee: Damien Katz
> Fix For: 0.9
>
>
> CouchDB is has been working 3 days without stop. I tried to insert large amount of data. From time to time I was doing compactation.
> One when database has 50GB of size and 3 mln records, then tried second time when there was 4,7 mln records and 12 GB but something failed. I tried to restart compaction from futon interface but it wasn't working.
> On console where was working inserting script written in erlang I received message.
> Err add {event,{session,965,3,2005718},
> {session_data,os,"Ubuntu 8.10",icecast,"WinampMPEG/5.53"},
> {listener,999999999999999,12301897,"85.112.221.200"},
> 1213297703,undefined,21,undefined,undefined} {ok,
> {{"HTTP/1.1",
> 500,
> "Internal Server Error"},
> [{"cache-control",
> "must-revalidate"},
> {"date",
> "Fri, 31 Oct 2008 19:13:43 GMT"},
> {"server",
> "CouchDB/0.8.1-incubating (Erlang OTP/R12B)"},
> {"content-length",
> "1665"},
> {"content-type",
> "text/plain;charset=utf-8"}],
> "{\"error\":\"EXIT\",\"reason\":\"{{function_clause,\\n [{couch_db,'-copy_docs\\/3-fun-0-',\\n [not_found,<0.24367.262>,\\n {stream,<0.25264.262>,<0.25263.262>},\\n <0.25263.262>]},\\n {lists,map,2},\\n {lists,map,2},\\n {couch_db,copy_docs,3},\\n {couch_db,'-copy_compact_docs\\/2-fun-0-',4},\\n {couch_btree,stream_kv_node2,7},\\n {couch_btree,stream_kp_node,6},\\n {couch_btree,stream_kp_node,7}]},\\n {gen_server,call,\\n [<0.24368.262>,\\n {update_docs,\\n [[{doc,\\\"965:3:2005718\\\",\\n [\\\"102178913\\\"],\\n {obj,\\n [{\\\"type\\\",\\\"event\\\"},\\n {\\\"session\\\",\\n {obj,\\n [{\\\"server_id\\\",965},\\n {\\\"instance_id\\\",3},\\n {\\\"id\\\",2005718}]}},\\n {\\\"session_data\\\",\\n {obj,\\n [{\\\"origin\\\",\\\"os\\\"},\\n {\\\"os_type\\\",\\\"Ubuntu 8.10\\\"},\\n {\\\"server_type\\\",\\\"icecast\\\"},\\n {\\\"player\\\",\\\"WinampMPEG\\/5.53\\\"}]}},\\n {\\\"listener\\\",\\n {obj,\\n [{\\\"pid\\\",999999999999999},\\n {\\\"gg\\\",12301897},\\n {\\\"ip\\\",\\\"85.112.221.200\\\"}]}},\\n {\\\"start_time\\\",1213297703},\\n {\\\"end_time\\\",\\\"undefined\\\"},\\n {\\\"channel\\\",21},\\n {\\\"intro\\\",\\\"undefined\\\"}]},\\n [],false,[]}]],\\n [new_edits]},\\n infinity]}}\"}"}}
> I think that the most important part is error reason:
> "{\"error\":\"EXIT\",\"reason\":\"{{function_clause,\\n [{couch_db,'-copy_docs\\/3-fun-0-',\\n [not_found,<0.24367.262>,\\n {stream,<0.25264.262>,<0.25263.262>},\\n <0.25263.262>]},\\n {lists,map,2},\\n {lists,map,2},\\n {couch_db,copy_docs,3},\\n {couch_db,'-copy_compact_docs\\/2-fun-0-',4},\\n {couch_btree,stream_kv_node2,7},\\n {couch_btree,stream_kp_node,6},\\n {couch_btree,stream_kp_node,7}]},\\n {gen_server,call,\\n [<0.24368.262>,\\n {update_docs,\\n [[{doc,\\\"965:3:2005718\\\",\\n [\\\"102178913\\\"],\\n {obj,\\n [{\\\"type\\\",\\\"event\\\"},\\n {\\\"session\\\",\\n {obj,\\n [{\\\"server_id\\\",965},\\n
> or second time
> "{\"error\":\"EXIT\",\"reason\":\"{noproc,{gen_server,call,\\n [<0.25341.262>,{pread_bin,18695772967},infinity]}}\"}"}
> It is strange that compaction alway failed when *.compact file has 2116687115 or 2116963527 bytes of size. Always approximately 2GB.
> Is it bug? or there is some config parameter which says about result compact file size. If here is one, why file before compaction can has 50 GB size?
> Im sory for my poor English.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.