You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Paul Joseph Davis (JIRA)" <ji...@apache.org> on 2009/10/07 03:48:31 UTC
[jira] Created: (COUCHDB-520) Make view indexing commit checkpoints
more often
Make view indexing commit checkpoints more often
------------------------------------------------
Key: COUCHDB-520
URL: https://issues.apache.org/jira/browse/COUCHDB-520
Project: CouchDB
Issue Type: Bug
Components: JavaScript View Server
Affects Versions: 0.11
Reporter: Paul Joseph Davis
Add commit headers to indexes when they are huge. This will make indexing huge databases from scratch suck less.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-520) Make view indexing commit
checkpoints more often
Posted by "Adam Kocoloski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763076#action_12763076 ]
Adam Kocoloski commented on COUCHDB-520:
----------------------------------------
Can you say more about this, Paul? We're already doing checkpoints every time we flush the view results to disk. What sucks about indexing huge databases from scratch?
> Make view indexing commit checkpoints more often
> ------------------------------------------------
>
> Key: COUCHDB-520
> URL: https://issues.apache.org/jira/browse/COUCHDB-520
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 0.11
> Reporter: Paul Joseph Davis
>
> Add commit headers to indexes when they are huge. This will make indexing huge databases from scratch suck less.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (COUCHDB-520) Make view indexing commit checkpoints
more often
Posted by "Adam Kocoloski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adam Kocoloski closed COUCHDB-520.
----------------------------------
Resolution: Fixed
Fix Version/s: 0.11
Assignee: Paul Joseph Davis
Paul found and fixed a regression on the view index checkpoints in r824968
> Make view indexing commit checkpoints more often
> ------------------------------------------------
>
> Key: COUCHDB-520
> URL: https://issues.apache.org/jira/browse/COUCHDB-520
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 0.11
> Reporter: Paul Joseph Davis
> Assignee: Paul Joseph Davis
> Fix For: 0.11
>
>
> Add commit headers to indexes when they are huge. This will make indexing huge databases from scratch suck less.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-520) Make view indexing commit
checkpoints more often
Posted by "Adam Kocoloski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763193#action_12763193 ]
Adam Kocoloski commented on COUCHDB-520:
----------------------------------------
Good idea. Now I really wish I had saved my .view file when it happened to me, but it was ~10 GB and built using a funky branch of couch so I didn't.
> Make view indexing commit checkpoints more often
> ------------------------------------------------
>
> Key: COUCHDB-520
> URL: https://issues.apache.org/jira/browse/COUCHDB-520
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 0.11
> Reporter: Paul Joseph Davis
>
> Add commit headers to indexes when they are huge. This will make indexing huge databases from scratch suck less.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-520) Make view indexing commit
checkpoints more often
Posted by "Adam Kocoloski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763235#action_12763235 ]
Adam Kocoloski commented on COUCHDB-520:
----------------------------------------
I tried playing those games, but Couch is pretty resilient. Every time it pops back up with a valid header.
> Make view indexing commit checkpoints more often
> ------------------------------------------------
>
> Key: COUCHDB-520
> URL: https://issues.apache.org/jira/browse/COUCHDB-520
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 0.11
> Reporter: Paul Joseph Davis
>
> Add commit headers to indexes when they are huge. This will make indexing huge databases from scratch suck less.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-520) Make view indexing commit
checkpoints more often
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763186#action_12763186 ]
Paul Joseph Davis commented on COUCHDB-520:
-------------------------------------------
Huh. I read through this code again this morning and you're right. I completely missed that partial_update was doing a send back for delayed_commit.
The weird error with timeouts on trying to read a header from these index files worries me though. If Glenn's indexing fails again I'll try seek through that file to figure out what's going on.
> Make view indexing commit checkpoints more often
> ------------------------------------------------
>
> Key: COUCHDB-520
> URL: https://issues.apache.org/jira/browse/COUCHDB-520
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 0.11
> Reporter: Paul Joseph Davis
>
> Add commit headers to indexes when they are huge. This will make indexing huge databases from scratch suck less.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-520) Make view indexing commit
checkpoints more often
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763206#action_12763206 ]
Paul Joseph Davis commented on COUCHDB-520:
-------------------------------------------
I'll try and reproduce later by 'kill -KILL'ing CouchDB durring an initial view write to see what happens. If my theory is right, I'll either get a no header error, or a header with 0 updates if we write one at the top of the file.
> Make view indexing commit checkpoints more often
> ------------------------------------------------
>
> Key: COUCHDB-520
> URL: https://issues.apache.org/jira/browse/COUCHDB-520
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 0.11
> Reporter: Paul Joseph Davis
>
> Add commit headers to indexes when they are huge. This will make indexing huge databases from scratch suck less.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.