You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Filipe Manana (Created) (JIRA)" <ji...@apache.org> on 2011/10/17 13:34:10 UTC
[jira] [Created] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
File descriptor leaks on design document update and view cleanup
----------------------------------------------------------------
Key: COUCHDB-1309
URL: https://issues.apache.org/jira/browse/COUCHDB-1309
Project: CouchDB
Issue Type: Bug
Reporter: Filipe Manana
Assignee: Filipe Manana
If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Filipe Manana (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Filipe Manana updated COUCHDB-1309:
-----------------------------------
Attachment: master-0002-Shutdown-view-group-on-ddoc-update.patch
master-0001-Add-ddoc_updated-event.patch
12x-0002-Shutdown-view-group-on-ddoc-update.patch
12x-0001-Add-ddoc_updated-event.patch
Updated patches to reuse the update notifier from couch_view (1.2.x) / couch_index_server (master)
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: 12x-0001-Add-ddoc_updated-event.patch, 12x-0002-Shutdown-view-group-on-ddoc-update.patch, couchdb-1309_12x.patch, couchdb-1309_trunk.patch, master-0001-Add-ddoc_updated-event.patch, master-0002-Shutdown-view-group-on-ddoc-update.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Filipe Manana (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Filipe Manana resolved COUCHDB-1309.
------------------------------------
Resolution: Fixed
Fix Version/s: 1.2
Applied to master and branch 1.2.x
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
> Attachments: 12x-0001-Add-ddoc_updated-event.patch, 12x-0002-Shutdown-view-group-on-ddoc-update.patch, couchdb-1309_12x.patch, couchdb-1309_trunk.patch, master-0001-Add-ddoc_updated-event.patch, master-0002-Shutdown-view-group-on-ddoc-update.patch, master-0002-Shutdown-view-group-on-ddoc-update.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Filipe Manana (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Filipe Manana updated COUCHDB-1309:
-----------------------------------
Attachment: 0001-Fix-ets-lookup-case_clause-error-after-ddoc-is-updat.patch
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
> Attachments: 0001-Fix-ets-lookup-case_clause-error-after-ddoc-is-updat.patch, 12x-0001-Add-ddoc_updated-event.patch, 12x-0002-Shutdown-view-group-on-ddoc-update.patch, couchdb-1309_12x.patch, couchdb-1309_trunk.patch, master-0001-Add-ddoc_updated-event.patch, master-0002-Shutdown-view-group-on-ddoc-update.patch, master-0002-Shutdown-view-group-on-ddoc-update.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Jan Lehnardt (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129658#comment-13129658 ]
Jan Lehnardt commented on COUCHDB-1309:
---------------------------------------
@Benoit, wouldn't this send a notification to every updater in every database every time a single design doc gets updated?
I agree it is less code, but it sure sounds like a very entangled result when a ddoc in database A changes and the updater for a ddoc in database B receives the notification. Especially in a situation with a lot of databases (e.g. a db per user) and design docs in each.
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Benoit Chesneau (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128811#comment-13128811 ]
Benoit Chesneau commented on COUCHDB-1309:
------------------------------------------
I'm not sure it need to be asynchronous. Couldn't we test it here :
http://git-wip-us.apache.org/repos/asf?p=couchdb.git;a=blob;f=src/couch_index/src/couch_index_updater.erl;h=853f3d11130c903650450e97b23a692762b3bba1;hb=HEAD#l118
and then crash the updater?
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Filipe Manana (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129666#comment-13129666 ]
Filipe Manana commented on COUCHDB-1309:
----------------------------------------
Benoit, when I asked you to provide sample code is just because I wasn't understand your point. Not meant for you to take it as an offence, but as a better/more clear way to express your ideas.
Regarding the first solution, as Jan pointed out, your notifying every index about every ddoc update.
To avoid this, one of the ets tables (BY_SIG maybe) should also store the ID of the ddoc associated to the indexer.
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Paul Joseph Davis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129056#comment-13129056 ]
Paul Joseph Davis commented on COUCHDB-1309:
--------------------------------------------
Filipe,
Good call. That code I'm remember is probably from before we used signatures for view groups instead of just the design name.
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Paul Joseph Davis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13131954#comment-13131954 ]
Paul Joseph Davis commented on COUCHDB-1309:
--------------------------------------------
I've been thinking about this and it occurs to me that adding the new function to the index seems like the wrong way to go about this. Seems to me that couch_index_server and couch_index should take care of all of this without requiring each index to reimplement this simple code. I'll take a quick pass through later to try and sketch out what i mean.
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Filipe Manana (Reopened) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Filipe Manana reopened COUCHDB-1309:
------------------------------------
Found a race condition that can happen and results in a case_clause error inside the view manager (couch_view / couch_index_server (master)).
Patch attached, 1.2.0 blocker.
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
> Attachments: 0001-Fix-ets-lookup-case_clause-error-after-ddoc-is-updat.patch, 12x-0001-Add-ddoc_updated-event.patch, 12x-0002-Shutdown-view-group-on-ddoc-update.patch, couchdb-1309_12x.patch, couchdb-1309_trunk.patch, master-0001-Add-ddoc_updated-event.patch, master-0002-Shutdown-view-group-on-ddoc-update.patch, master-0002-Shutdown-view-group-on-ddoc-update.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Paul Joseph Davis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149361#comment-13149361 ]
Paul Joseph Davis commented on COUCHDB-1309:
--------------------------------------------
So reviewing this again, my last comment still stands. There's no reason to introduce the ddoc_changed function into the indexer API. These changes should be limited entirely to couch_index (minus the tests anyway). This is a simple fix. Just use Mod:get(signature, Mod:init(Db, DDoc)) (or similar) in the with_db call in couch_index:handle_cast/2 call for ddoc_updated.
The other thing that occurred to me over lunch is if we might want to reimplement this by just having couch_index close itself after a period of inactivity. At the moment there's no limit on the number of groups that are opened like we have for databases. Some sort of mechanism to close these things down after lack of use seems like a good idea. The patch to do this would be pretty trivial. All you have to do is add a Timeout to each reply/noreply response in the couch_index gen_server, and then have a handle_info(timeout) clause that returns {stop, normal, State}.
Other than that, I've heard other people wanting the update notifications for design docs so that could go in regardless. Another thing that we might want to think about is best practices for commits and whether commits should span applications. I don't have super strong feelings either way though I have heard the argument that not spanning applications decreases the chance that code starts getting super entangled.
Good work, Filipe
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: 12x-0001-Add-ddoc_updated-event.patch, 12x-0002-Shutdown-view-group-on-ddoc-update.patch, couchdb-1309_12x.patch, couchdb-1309_trunk.patch, master-0001-Add-ddoc_updated-event.patch, master-0002-Shutdown-view-group-on-ddoc-update.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Paul Joseph Davis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129120#comment-13129120 ]
Paul Joseph Davis commented on COUCHDB-1309:
--------------------------------------------
@Filipe
After reading further I think you've probably got the best approach. The only alternative I've contemplated would be to have a monitor process that wakes up every so often and checks that its sig is still valid which I don't think is as clean as this version.
Although, I would like to see this patch as at least two if not more commits. Specifically, at least one for the new ddoc event broadcasting and then one for the indexer/view stuff.
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Filipe Manana (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128814#comment-13128814 ]
Filipe Manana commented on COUCHDB-1309:
----------------------------------------
Not sure what you mean:
"I'm not sure it need to be asynchronous. Couldn't we test it here : "
It's synchronous to avoid getting the process' mailbox flooded in the unlikely case there's a high rate of updates to the same ddoc.
And the updater shouldn't be crashed. If it's running it means there are clients waiting for it before folding the view. Those clients should not get an error.
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Filipe Manana (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128936#comment-13128936 ]
Filipe Manana commented on COUCHDB-1309:
----------------------------------------
Benoit, what I meant by synchronous is the handle_call vs handle_cast in couch_index.erl.
I'm afraid you're missing some details about how the view system works. There's more than the updater process that depends on the design document properties and view signature, like couch_index_server.
Spawning a new group when the ddoc signature changes is not a new behaviour I'm adding - it was always like this afaik.
I'm concentrated on fixing this particular issue and not redesigning a big part of the view system. Feel free to do it and provide a working patch.
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Paul Joseph Davis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129042#comment-13129042 ]
Paul Joseph Davis commented on COUCHDB-1309:
--------------------------------------------
Hrm. Good catch Filipe.
I haven't done more than skim the patch, but I'm wondering if it might be easier to just shutdown the view group when it sees the new design doc during an update instead of adding all of the mechanics to listen for update notifications. I'll read through this more when I get done figuring out how deb packages work.
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Filipe Manana (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Filipe Manana resolved COUCHDB-1309.
------------------------------------
Resolution: Fixed
Fix applied to 1.2.x and master.
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Fix For: 1.2
>
> Attachments: 0001-Fix-ets-lookup-case_clause-error-after-ddoc-is-updat.patch, 12x-0001-Add-ddoc_updated-event.patch, 12x-0002-Shutdown-view-group-on-ddoc-update.patch, couchdb-1309_12x.patch, couchdb-1309_trunk.patch, master-0001-Add-ddoc_updated-event.patch, master-0002-Shutdown-view-group-on-ddoc-update.patch, master-0002-Shutdown-view-group-on-ddoc-update.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Benoit Chesneau (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129039#comment-13129039 ]
Benoit Chesneau commented on COUCHDB-1309:
------------------------------------------
Sending a synchronous call doesn't mean it need to stay alive, just that we send a message to the parent synchronously before dying. On the other hand it allows to not spawn another process. A custom exit signal could be enough.
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Filipe Manana (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129025#comment-13129025 ]
Filipe Manana commented on COUCHDB-1309:
----------------------------------------
Yes, that's assuming the updater will run again after the design document update. What if it doesn't? You still end up in the current situation.
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Benoit Chesneau (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129080#comment-13129080 ]
Benoit Chesneau commented on COUCHDB-1309:
------------------------------------------
I'm pretty sure things could be handled with a less offensive tone. Thanks. Tickets are here to let other review.
Like I said I will have a closer look *later*. I don't think we need another asynchronous process and more I'm not sure for a performance reason that the change in update_notifier is really wanted.
Even if we take this patch the notifier may need to be monitored, etc.
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Filipe Manana (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Filipe Manana updated COUCHDB-1309:
-----------------------------------
Attachment: couchdb-1309_12x.patch
couchdb-1309_trunk.patch
the patches for trunk and 1.2.x/1.1.x
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Benoit Chesneau (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129520#comment-13129520 ]
Benoit Chesneau commented on COUCHDB-1309:
------------------------------------------
Ok, the update detection can't happen in updater, if this isn't triggered.
Anyway I think we could remove this other listener by reusing the update notification process in couch_index_server, something like (not tested):
http://friendpaste.com/7CwKM5clixdyuq6HWHjDDv
I'm also not sure we need to know the exact design id, isn't emitting an 'refresh_indexes' event enough since we have all the info in states to detect signature change if it happends? It woud remove this list in memory and extra event notifications. Something like :
http://friendpaste.com/44q84nzKSjaU15vX2sVkfF
I will try to send a complete patch later in the day, but not sure I will have time. Btw I'm only worrying about the trunk here. I think we have some time to solve this problem on trunk. No need for a quick fix.
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Jan Lehnardt (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139323#comment-13139323 ]
Jan Lehnardt commented on COUCHDB-1309:
---------------------------------------
@Paul, sounds like an optimisation to me, would it make sense to get Filipe's patch in and then open a new ticket for your approach?
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Filipe Manana (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129046#comment-13129046 ]
Filipe Manana commented on COUCHDB-1309:
----------------------------------------
Benoit, you can't rely on the updater to detect that the ddoc changed.
Maybe I wasn't clear enough before. Imagine the following scenario:
1) Create a ddoc
2) Query one of its views
3) Update the ddoc
If all subsequent view query requests arrive after the update (3), they will get routed to the new view group - therefore the old one will not get its updater running again and will stay alive forever.
Same issue happens if when the update happens clients who get "into" the old view group are querying only with ?stale=ok.
If you're so convinced that doing it in the updater works for these 2 cases, please provide a working code prototype.
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Benoit Chesneau (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128985#comment-13128985 ]
Benoit Chesneau commented on COUCHDB-1309:
------------------------------------------
Or maybe I'm not. I'm not speaking about redesigning how the view system works neither I don't want to fix this issue and I'm not sure why you are suggesting that.
I'm trying to remove the need of this another asynchronous process. You don't really need it. Once the updater detect that this particular design doc has changed (and it will) it can simply send a synchronous signal to its parent. then the parent will do exactly the same you are doing in your patch. That's all. I don't see why it couldn't work. I will have a closer look later on it.
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Filipe Manana (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Filipe Manana updated COUCHDB-1309:
-----------------------------------
Attachment: master-0002-Shutdown-view-group-on-ddoc-update.patch
Thanks Paul.
Yes, that point was not addressed as I wasn't seeing what your idea was. You were supposed to explain it, according to the previous comment, but you forgot to do it until now.
About the shutdown after an inactivity period, I considered that some time ago (I think we even discussed that on irc). However that's a separate issue.
Updated patch master-0002-Shutdown-view-group-on-ddoc-update.patch
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: 12x-0001-Add-ddoc_updated-event.patch, 12x-0002-Shutdown-view-group-on-ddoc-update.patch, couchdb-1309_12x.patch, couchdb-1309_trunk.patch, master-0001-Add-ddoc_updated-event.patch, master-0002-Shutdown-view-group-on-ddoc-update.patch, master-0002-Shutdown-view-group-on-ddoc-update.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Benoit Chesneau (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129045#comment-13129045 ]
Benoit Chesneau commented on COUCHDB-1309:
------------------------------------------
@davisp mostly what I had in head.
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1309) File descriptor leaks on design
document update and view cleanup
Posted by "Benoit Chesneau (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128841#comment-13128841 ]
Benoit Chesneau commented on COUCHDB-1309:
------------------------------------------
using the db update notifier is an asynchronous process fpr me ...
By crash I mean in the OTP way. ie crash the updater then the manager restart one with new ddoc. That what you do anyway but it could be done directly in the updater, the manager should take care of such event.
> File descriptor leaks on design document update and view cleanup
> ----------------------------------------------------------------
>
> Key: COUCHDB-1309
> URL: https://issues.apache.org/jira/browse/COUCHDB-1309
> Project: CouchDB
> Issue Type: Bug
> Reporter: Filipe Manana
> Assignee: Filipe Manana
> Attachments: couchdb-1309_12x.patch, couchdb-1309_trunk.patch
>
>
> If we add a design document with views defined in it, open the corresponding view group (by querying one of its views for e.g.), then update the design document in such a way that the view signature changes (changing a view's map function code for e.g), the old view group remains open forever (unless a server restart happens) and keeps it's view file reference counter active forever.
> If a view cleanup request comes, the old view file is not deleted since the reference counter is not dropped by the old view group, keeping the file descriptor in use forever.
> This leakage is different from what is reported in COUCHDB-1129 but it's somehow related.
> The attached patch, simply shutdowns a view group when the design document is updated and the new view signature changes, releasing the old view file descriptor (as soon as no more clients are using the old view).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira