You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by GitBox <gi...@apache.org> on 2018/04/06 12:38:00 UTC
[GitHub] olivenvrac opened a new issue #1267: [ERROR] fabric_worker_timeout
olivenvrac opened a new issue #1267: [ERROR] fabric_worker_timeout
URL: https://github.com/apache/couchdb/issues/1267
I think that I have the same issue than this one : Various error messages and breakage #859
I open a new issue because i have somes questions about this.
SO, i'm getting the following errors which I don't know how to properly interpret and resolve too : "fabric_worker_timeout changes"
Also, i don't know how to reproduce these errors, and I am not sure what it is about and the consequences
I'd never had this probleme before.
after reading some comments about this, i have set the parameter "request_timeout" at "infinity" in the local.ini (section [fabric] and restart the CouchDB instance, wich seems to resolve my probleme
after this setting, the relplication between pouchdb and couchdb working again
For information, i've also use a continous replication on the CouchDB instance on wich i have some problems with and another CouchDB instance (using the _replicator db).
This replication works properly during the probleme of replication beetwen PouchDB and CouchDB.
But the problem with the fabric_worker_timeout persist even if i stop the replication under the couchdb instances by suppress the _replicator documents.
## Context
We are using PouchDb on an mobile application, which synchronyse his documents and attachments with a CouchDB.
Couchdb is configurated in standalone mode
PouchDB initiate the replication.
only 9 users connected with a poor amount of data, but some attachements in docs (a few MB)
## Current Behavior
Here are the errors in the couchdb.log after a GET on a document by PouchDB :
`[notice] 2018-04-05T15:05:41.934621Z couchdb@server.domain <0.577.3> 4c082f6d6f serveur:5984 xx.145.104.104 undefined GET //mydb/_local/xBlBiDk7hOBqC6G6Lc6TtQ%3D%3D?&_nonce=152294074184`
`[error] 2018-04-05T15:05:43.123458Z couchdb@server.domain <0.30455.2> 19157d72c9 fabric_worker_timeout changes,'couchdb@server.domain',<<"shards/c0000000-dfffffff/mydb.1513246909">>`
`[notice] 2018-04-05T15:05:43.124306Z couchdb@server.domain <0.30455.2> 19157d72c9 cuzarect01.orleanssrv.domsma:5984 xx.145.104.104 undefined GET //mydb/_changes?timeout=25000&style=all_docs&filter=filters%2Fus
er&user=a16851&since=5911g1AAAAMfeJy1zc0NgkAQBeAJELULjQUQxF1YTtKJ7g8JIQsYiFy1Db1oJ9qJdoLLQuQqCVzeJ...............h5U9brlU5-QLTLv91&limit=6&_nonce=1522940683161 500 ok 600137 404 ok 46`
`[error] 2018-04-05T15:05:43.515755Z couchdb@server.domain <0.15633.1> d73f53a063 rexi_server exit:timeout [{rexi,init_stream,1,[{file,"src/rexi.erl"},{line,256}]},{rexi,stream2,3,[{file,"src/rexi.erl"},{line,204}]},{fabric_rpc,changes_enumerator,2,[{file,"src/fabric_rpc.erl"},{line,349}]},{couch_btree,stream_kv_node2,8,[{file,"src/couch_btree.erl"},{line,783}]},{couch_btree,stream_kp_node,7,[{file,"src/couch_btree.erl"},{line,710}]},{couch_btree,stream_kp_node,8,[{file
,"src/couch_btree.erl"},{line,754}]},{couch_btree,fold,4,[{file,"src/couch_btree.erl"},{line,220}]},{couch_db,changes_since,5,[{file,"src/couch_db.erl"},{line,1244}]}]`
## Questions :
is the cpu overloaded must be the cause the problem or the problem cause the overloaded
is this errors are induced by the request on the handler "_changes" or "_local" ?
is it possible that this problem must comes from a slowly network between pouchdb and couchdb (3G connexion)?
in this case, is it rather PouchDb that must catch some errors and not on the CouchDB server ?
is the parameter request_timeout = infinity is a good approach ? I suppose it's beter to determine a correct value higher than de default value.
You'v writting that using filtered _changes feeds could relate to your couchjs os process timeouts. You probably right ! Indeed, the replication initiate by PouchDB use filters changes feeds.
here is the js function : function(doc,req) { return doc.user === req.query.user || doc._id === '_design/filters' }
Must I switch this to Mango-style selectors which are significantly better performance than JavaScript filters as you say ?
## Environment
version os and couchdb :
Linux version 3.10.0-327.el7.x86_64 (RHEL 7)
CouchDB 2.0
ulimit :
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 7249
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 4096
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
CouchDB configuration :
{
"native_query_servers": {
"query": "{mango_native_proc, start_link, []}"
},
"httpd_design_handlers": {
"_compact": "{couch_mrview_http, handle_compact_req}",
"_info": "{couch_mrview_http, handle_info_req}",
"_list": "{couch_mrview_show, handle_view_list_req}",
"_rewrite": "{couch_httpd_rewrite, handle_rewrite_req}",
"_show": "{couch_mrview_show, handle_doc_show_req}",
"_update": "{couch_mrview_show, handle_doc_update_req}",
"_view": "{couch_mrview_http, handle_view_req}",
"_view_changes": "{couch_mrview_http, handle_view_changes_req}"
},
"uuids": {
"algorithm": "sequential",
"max_count": "1000"
},
"cluster": {
"n": "3",
"q": "8"
},
"fabric": {
"request_timeout": "infinity"
},
"cors": {
"credentials": "true",
"headers": "accept, authorization, content-type, origin, referer",
"methods": "GET, PUT, POST, HEAD, DELETE",
"origins": "*"
},
"chttpd": {
"backlog": "512",
"bind_address": "0.0.0.0",
"docroot": "./share/www",
"port": "5984",
"socket_options": "[{recbuf, 262144}, {sndbuf, 262144}, {nodelay, true}]"
},
"httpd_global_handlers": {
"/": "{couch_httpd_misc_handlers, handle_welcome_req, <<\"Welcome\">>}",
"_active_tasks": "{couch_httpd_misc_handlers, handle_task_status_req}",
"_all_dbs": "{couch_httpd_misc_handlers, handle_all_dbs_req}",
"_config": "{couch_httpd_misc_handlers, handle_config_req}",
"_oauth": "{couch_httpd_oauth, handle_oauth_req}",
"_plugins": "{couch_plugins_httpd, handle_req}",
"_replicate": "{couch_replicator_httpd, handle_req}",
"_restart": "{couch_httpd_misc_handlers, handle_restart_req}",
"_session": "{couch_httpd_auth, handle_session_req}",
"_stats": "{couch_stats_httpd, handle_stats_req}",
"_system": "{chttpd_misc, handle_system_req}",
"_utils": "{couch_httpd_misc_handlers, handle_utils_dir_req, \"./share/www\"}",
"_uuids": "{couch_httpd_misc_handlers, handle_uuids_req}",
"favicon.ico": "{couch_httpd_misc_handlers, handle_favicon_req, \"./share/www\"}"
},
"attachments": {
"compressible_types": "text/*, application/javascript, application/json, application/xml",
"compression_level": "8"
},
"admins": {
"cuadmin": "-pbkdf2-ab0***************************************************,10"
},
"query_server_config": {
"os_process_limit": "25",
"reduce_limit": "true"
},
"vendor": {
"name": "The Apache Software Foundation"
},
"replicator": {
"connection_timeout": "30000",
"http_connections": "20",
"max_replication_retry_count": "10",
"retries_per_request": "10",
"socket_options": "[{keepalive, true}, {nodelay, false}]",
"ssl_certificate_max_depth": "3",
"start_delay": "0",
"start_splay": "0",
"verify_ssl_certificates": "false",
"worker_batch_size": "500",
"worker_processes": "4"
},
"couch_httpd_oauth": {
"use_users_db": "false"
},
"ssl": {
"port": "6984"
},
"log": {
"file": "./var/log/couchdb.log",
"level": "info",
"writer": "file"
},
"indexers": {
"couch_mrview": "true"
},
"view_compaction": {
"keyvalue_buffer_size": "2097152"
},
"query_servers": {
"coffeescript": "./bin/couchjs ./share/server/main-coffee.js",
"javascript": "./bin/couchjs ./share/server/main.js"
},
"daemons": {
"auth_cache": "{couch_auth_cache, start_link, []}",
"compaction_daemon": "{couch_compaction_daemon, start_link, []}",
"couch_peruser": "{couch_peruser, start_link, []}",
"external_manager": "{couch_external_manager, start_link, []}",
"httpd": "{couch_httpd, start_link, []}",
"index_server": "{couch_index_server, start_link, []}",
"os_daemons": "{couch_os_daemons, start_link, []}",
"query_servers": "{couch_proc_manager, start_link, []}",
"uuids": "{couch_uuids, start, []}",
"vhosts": "{couch_httpd_vhost, start_link, []}"
},
"couch_peruser": {
"delete_dbs": "false",
"enable": "false"
},
"httpd": {
"allow_jsonp": "true",
"authentication_handlers": "{couch_httpd_oauth, oauth_authentication_handler}, {couch_httpd_auth, cookie_authentication_handler}, {couch_httpd_auth, default_authentication_handler}",
"bind_address": "127.0.0.1",
"default_handler": "{couch_httpd_db, handle_request}",
"enable_cors": "true",
"port": "5986",
"secure_rewrites": "true",
"socket_options": "[{recbuf, 262144}, {sndbuf, 262144}]",
"vhost_global_handlers": "_utils, _uuids, _session, _oauth, _users"
},
"httpd_db_handlers": {
"_all_docs": "{couch_mrview_http, handle_all_docs_req}",
"_changes": "{couch_httpd_db, handle_db_changes_req}",
"_compact": "{couch_httpd_db, handle_compact_req}",
"_design": "{couch_httpd_db, handle_design_req}",
"_design_docs": "{couch_mrview_http, handle_design_docs_req}",
"_local_docs": "{couch_mrview_http, handle_local_docs_req}",
"_temp_view": "{couch_mrview_http, handle_temp_view_req}",
"_view_cleanup": "{couch_mrview_http, handle_cleanup_req}"
},
"database_compaction": {
"checkpoint_after": "5242880",
"doc_buffer_size": "524288"
},
"csp": {
"enable": "true"
},
"couch_httpd_auth": {
"allow_persistent_cookies": "false",
"auth_cache_size": "50",
"authentication_db": "_users",
"authentication_redirect": "/_utils/session.html",
"iterations": "10",
"require_valid_user": "false",
"secret": "*********************",
"timeout": "7200"
},
"couchdb": {
"attachment_stream_buffer_size": "4096",
"database_dir": "../r1/data",
"default_security": "admin_local",
"delayed_commits": "false",
"file_compression": "snappy",
"max_dbs_open": "500",
"max_document_size": "67108864",
"os_process_timeout": "5000",
"uuid": "*************************",
"view_index_dir": "../r1/data"
},
"compaction_daemon": {
"check_interval": "300",
"min_file_size": "131072"
}
}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services