You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Jan Lehnardt (Created) (JIRA)" <ji...@apache.org> on 2012/03/18 21:36:42 UTC
[jira] [Created] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
CouchDB deletes .view files if it can't open them, even if the error is "emfile".
---------------------------------------------------------------------------------
Key: COUCHDB-1445
URL: https://issues.apache.org/jira/browse/COUCHDB-1445
Project: CouchDB
Issue Type: Bug
Components: JavaScript View Server
Affects Versions: 1.2
Reporter: Jan Lehnardt
Fix For: 1.2
Via Stefan Kögl on dev@:
Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
live traffic to the instance and after a rather short time, requests
were failing with the following information in the logs:
[Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
{error_report,<0.31.0>,
{<0.27554.2>,std_error,
[{application,mochiweb},
"Accept failed error",
"{error,emfile}"]}}
[Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
{error_report,<0.31.0>,
{<0.27554.2>,crash_report,
[[{initial_call,
{mochiweb_acceptor,init,
['Argument__1','Argument__2',
'Argument__3']}},
{pid,<0.27554.2>},
{registered_name,[]},
{error_info,
{exit,
{error,accept_failed},
[{mochiweb_acceptor,init,3},
{proc_lib,init_p_do_apply,3}]}},
{ancestors,
[couch_httpd,couch_secondary_services,
couch_server_sup,<0.32.0>]},
{messages,[]},
{links,[<0.129.0>]},
{dictionary,[]},
{trap_exit,false},
{status,running},
{heap_size,233},
{stack_size,24},
{reductions,244}],
[]]}}
I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
more files / connections. I've set the (hard and soft) nofile limit for
user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
Anyway, as soon as the error occured, CouchDB started writing most of my
view files from scratch, rendering the instance unusable.
I'd expect CouchDB to fail more gracefully when the maximum number of
open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Robert Newson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Newson resolved COUCHDB-1445.
------------------------------------
Resolution: Fixed
Randall resolved this a few days ago but forgot to close the ticket like the naughty person he is.
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.1.1, 1.3
> Reporter: Jan Lehnardt
> Assignee: Randall Leeds
> Priority: Blocker
> Fix For: 1.3, 1.2
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Robert Newson (Reopened) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Newson reopened COUCHDB-1445:
------------------------------------
Not fixed on master. I am trying to fix that but haven't succeeded yet (porting the test is fiddly).
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.1.1, 1.2, 1.3
> Reporter: Jan Lehnardt
> Assignee: Randall Leeds
> Priority: Blocker
> Fix For: 1.2, 1.3, 1.1.2
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Jan Lehnardt (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13232357#comment-13232357 ]
Jan Lehnardt commented on COUCHDB-1445:
---------------------------------------
@Randall & others: what other conditions like emfile can happen where creating the file from scratch is not the right cause of action?
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.2
> Reporter: Jan Lehnardt
> Fix For: 1.2
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Randall Leeds (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Randall Leeds closed COUCHDB-1445.
----------------------------------
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.1.1, 1.3
> Reporter: Jan Lehnardt
> Assignee: Randall Leeds
> Priority: Blocker
> Fix For: 1.2, 1.3
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Randall Leeds (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Randall Leeds updated COUCHDB-1445:
-----------------------------------
Affects Version/s: (was: 1.2)
Fix Version/s: (was: 1.2.1)
1.2
This was fixed before the 1.2 release. It's open still because it needs a forward-port, which I've committed myself to doing.
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.1.1, 1.3
> Reporter: Jan Lehnardt
> Assignee: Randall Leeds
> Priority: Blocker
> Fix For: 1.1.2, 1.2, 1.3
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Jan Lehnardt (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13232356#comment-13232356 ]
Jan Lehnardt commented on COUCHDB-1445:
---------------------------------------
Git blame leads to:
commit 56a3ee28e006aa42150482e1c3f91dc1906273f9
Author: Damien F. Katz <da...@apache.org>
Date: Fri Dec 12 05:23:37 2008 +0000
modifications to view server to keep the file descriptor open for the life of the view group.
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.2
> Reporter: Jan Lehnardt
> Fix For: 1.2
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Randall Leeds (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Randall Leeds reassigned COUCHDB-1445:
--------------------------------------
Assignee: Randall Leeds
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.2
> Reporter: Jan Lehnardt
> Assignee: Randall Leeds
> Fix For: 1.2
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Robert Newson (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Newson updated COUCHDB-1445:
-----------------------------------
Fix Version/s: 1.1.2
let's backport to 1.1.x just in case.
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.2
> Reporter: Jan Lehnardt
> Assignee: Randall Leeds
> Fix For: 1.2, 1.1.2
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Noah Slater (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Noah Slater updated COUCHDB-1445:
---------------------------------
Fix Version/s: (was: 1.2)
1.2.1
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.1.1, 1.2, 1.3
> Reporter: Jan Lehnardt
> Assignee: Randall Leeds
> Priority: Blocker
> Fix For: 1.2.1, 1.3, 1.1.2
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Benoit Chesneau (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13235578#comment-13235578 ]
Benoit Chesneau commented on COUCHDB-1445:
------------------------------------------
ok, i'm on it , will provide the patch this afternoon.
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.1.1, 1.2, 1.3
> Reporter: Jan Lehnardt
> Assignee: Randall Leeds
> Priority: Blocker
> Fix For: 1.2, 1.3, 1.1.2
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Paul Joseph Davis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13232401#comment-13232401 ]
Paul Joseph Davis commented on COUCHDB-1445:
--------------------------------------------
And then Randall reminds me its actually file:format_error. Note to future selves, don't bother trying to grep for strerror or perror in Erlang sources.
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.2
> Reporter: Jan Lehnardt
> Assignee: Randall Leeds
> Fix For: 1.2, 1.1.2
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Randall Leeds (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Randall Leeds updated COUCHDB-1445:
-----------------------------------
Priority: Blocker (was: Major)
Affects Version/s: 1.3
1.1.1
Fix Version/s: 1.3
Fixed on 1.1.x and 1.2.x. Blocking master for 1.3.
To https://git-wip-us.apache.org/repos/asf/couchdb.git
29eac04..94e72e7 1.2.x -> 1.2.x
785d32f..24a61fd 1.1.x -> 1.1.x
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.1.1, 1.2, 1.3
> Reporter: Jan Lehnardt
> Assignee: Randall Leeds
> Priority: Blocker
> Fix For: 1.2, 1.3, 1.1.2
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Paul Joseph Davis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13235361#comment-13235361 ]
Paul Joseph Davis commented on COUCHDB-1445:
--------------------------------------------
It'd probably have to be a manual merge of the diff from 1.2 since the view engine is so different.
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.1.1, 1.2, 1.3
> Reporter: Jan Lehnardt
> Assignee: Randall Leeds
> Priority: Blocker
> Fix For: 1.2, 1.3, 1.1.2
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Robert Newson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Newson updated COUCHDB-1445:
-----------------------------------
Fix Version/s: (was: 1.1.2)
Removing 1.0.4/1.1.2.
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.1.1, 1.3
> Reporter: Jan Lehnardt
> Assignee: Randall Leeds
> Priority: Blocker
> Fix For: 1.2, 1.3
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Paul Joseph Davis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13232398#comment-13232398 ]
Paul Joseph Davis commented on COUCHDB-1445:
--------------------------------------------
Yep. This is a pretty good explanation and solution. This behavior has been there forever. I always understood the intent to be what this ticket decided (rewriting views is fine, so just trying to blow them away and rewriting to solve errors is acceptable). I agree that we can switch this to a log and abort the open. The exact implementation of which can be in a couple places but I'm not overly tied to any particular one.
One caveat, I'm pretty sure there's a function that works out to be the equivalent of strerror. We should make use of that rather than trying to generate our own posix error messages.
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.2
> Reporter: Jan Lehnardt
> Assignee: Randall Leeds
> Fix For: 1.2, 1.1.2
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Paul Joseph Davis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13232400#comment-13232400 ]
Paul Joseph Davis commented on COUCHDB-1445:
--------------------------------------------
Wow. Took me a while to find it, but its (appropriately named) erl_posix_msg:message/1 which takes the reason from the posix tuple.
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.2
> Reporter: Jan Lehnardt
> Assignee: Randall Leeds
> Fix For: 1.2, 1.1.2
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Robert Newson (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Newson resolved COUCHDB-1445.
------------------------------------
Resolution: Fixed
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.1.1, 1.2, 1.3
> Reporter: Jan Lehnardt
> Assignee: Randall Leeds
> Priority: Blocker
> Fix For: 1.2, 1.3, 1.1.2
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Jan Lehnardt (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jan Lehnardt updated COUCHDB-1445:
----------------------------------
Comment: was deleted
(was: @Randall & others: what other conditions like emfile can happen where creating the file from scratch is not the right cause of action?)
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.2
> Reporter: Jan Lehnardt
> Assignee: Randall Leeds
> Fix For: 1.2, 1.1.2
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Jan Lehnardt (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13232358#comment-13232358 ]
Jan Lehnardt commented on COUCHDB-1445:
---------------------------------------
@Randall & others: what other conditions like emfile can happen where creating the file from scratch is not the right cause of action?
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.2
> Reporter: Jan Lehnardt
> Fix For: 1.2
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Jan Lehnardt (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13232355#comment-13232355 ]
Jan Lehnardt commented on COUCHDB-1445:
---------------------------------------
Randall Leeds replies:
Looks like a bug. Whenever there's a problem opening a view file,
couch_view tries to delete it. Clearly, this is not the right course of
action when the problem is due to emfile.
Here's a patch that I propose might fix it. I'd like to hear from another
dev on this, or if there's a better way we should bail out.
diff --git a/src/couchdb/couch_view_group.erl
b/src/couchdb/couch_view_group.erl
index 97fc512..ab075bd 100644
--- a/src/couchdb/couch_view_group.erl
+++ b/src/couchdb/couch_view_group.erl
@@ -469,6 +469,10 @@ open_index_file(RootDir, DbName, GroupSig) ->
case couch_file:open(FileName) of
{ok, Fd} -> {ok, Fd};
{error, enoent} -> couch_file:open(FileName, [create]);
+ {error, emfile} ->
+ ?LOG_ERROR("Could not open file for view index: max open files
reached. "
+ "Raise ERL_MAX_PORTS or system limits.", []),
+ throw({error, emfile});
Error -> Error
end.
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.2
> Reporter: Jan Lehnardt
> Fix For: 1.2
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Benoit Chesneau (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13235351#comment-13235351 ]
Benoit Chesneau commented on COUCHDB-1445:
------------------------------------------
That's here where I really like diff, instead having to go in another branch to find useful changes.
How do we proceed to add these changes to the HEAD ? Using 1.2 or 1.1 history ? 1 commit ?
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.1.1, 1.2, 1.3
> Reporter: Jan Lehnardt
> Assignee: Randall Leeds
> Priority: Blocker
> Fix For: 1.2, 1.3, 1.1.2
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1445) CouchDB deletes .view files if it
can't open them, even if the error is "emfile".
Posted by "Robert Newson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13232375#comment-13232375 ]
Robert Newson commented on COUCHDB-1445:
----------------------------------------
To summarize couchdb-dev IRC chat, we (Randall, Jan, myself) don't think there's any error condition when opening a view file that warrants blowing the view file away. The behaviour dates to 2008 where, perhaps, it was considered better to automatically recover from any error in a view file by starting over. CouchDB's view handling is well-tested now and so this pessimistic recovery mechanism is overly aggressive.
Instead, we will log an error to the log file and throw in the code.
> CouchDB deletes .view files if it can't open them, even if the error is "emfile".
> ---------------------------------------------------------------------------------
>
> Key: COUCHDB-1445
> URL: https://issues.apache.org/jira/browse/COUCHDB-1445
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Affects Versions: 1.2
> Reporter: Jan Lehnardt
> Assignee: Randall Leeds
> Fix For: 1.2, 1.1.2
>
>
> Via Stefan Kögl on dev@:
> Another thing I noticed during my tests of CouchDB 1.2.x. I redirected
> live traffic to the instance and after a rather short time, requests
> were failing with the following information in the logs:
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,std_error,
> [{application,mochiweb},
> "Accept failed error",
> "{error,emfile}"]}}
> [Sun, 18 Mar 2012 16:39:24 GMT] [error] [<0.27554.2>]
> {error_report,<0.31.0>,
> {<0.27554.2>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2',
> 'Argument__3']}},
> {pid,<0.27554.2>},
> {registered_name,[]},
> {error_info,
> {exit,
> {error,accept_failed},
> [{mochiweb_acceptor,init,3},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.32.0>]},
> {messages,[]},
> {links,[<0.129.0>]},
> {dictionary,[]},
> {trap_exit,false},
> {status,running},
> {heap_size,233},
> {stack_size,24},
> {reductions,244}],
> []]}}
> I think "emfile" means that CouchDB (or mochiweb?) couldn't open any
> more files / connections. I've set the (hard and soft) nofile limit for
> user couchdb to 4096, but didn't raise the ERL_MAX_PORTS accordingly.
> Anyway, as soon as the error occured, CouchDB started writing most of my
> view files from scratch, rendering the instance unusable.
> I'd expect CouchDB to fail more gracefully when the maximum number of
> open files is reached. Is this a bug or expected behaviour?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira