You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by GitBox <gi...@apache.org> on 2019/12/06 22:56:44 UTC
[GitHub] [couchdb] wohali opened a new issue #2344: [moved] Report of
CouchDB + k8s + NFS crashing
wohali opened a new issue #2344: [moved] Report of CouchDB + k8s + NFS crashing
URL: https://github.com/apache/couchdb/issues/2344
@nue-melexis asks:
Couchdb was running fine and recently stopped working. Also after reboot it crashes.
Logs from restart of container/pod
```
[error] 2019-12-05T08:37:28.628617Z nonode@nohost <0.386.0> -------- gen_server <0.386.0> terminated with reason: no match of right hand value eof at couch_file:read_raw_iolist_int/3(line:627) <= couch_file:handle_call/3(line:449) <= gen_server:try_handle_call/4(line:615) <= gen_server:handle_msg/5(line:647) <= proc_lib:init_p_do_apply/3(line:247)
last msg: {pread_iolist,12475}
state: [{data,[{"State",{file,{file_descriptor,prim_file,{#Port<0.5682>,33}},true,37059,#Ref<0.0.3.1196>,infinity}},{"InitialFilePath","./data/shards/a0000000-bfffffff/_users.1570022815.couch"}]}]
extra: []
[info] 2019-12-05T08:37:28.638598Z nonode@nohost <0.212.0> -------- db shards/a0000000-bfffffff/_users.1570022815 died with reason {{badmatch,eof},[{couch_file,read_raw_iolist_int,3,[{file,"src/couch_file.erl"},{line,627}]},{couch_file,handle_call,3,[{file,"src/couch_file.erl"},{line,449}]},{gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,615}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,647}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,247}]}]}
[error] 2019-12-05T08:37:28.639046Z nonode@nohost <0.386.0> -------- CRASH REPORT Process (<0.386.0>) with 1 neighbors exited with reason: no match of right hand value eof at couch_file:read_raw_iolist_int/3(line:627) <= couch_file:handle_call/3(line:449) <= gen_server:try_handle_call/4(line:615) <= gen_server:handle_msg/5(line:647) <= proc_lib:init_p_do_apply/3(line:247) at gen_server:terminate/7(line:812) <= proc_lib:init_p_do_apply/3(line:247); initial_call: {couch_file,init,['Argument__1']}, ancestors: [<0.385.0>,<0.384.0>], messages: [], links: [<0.385.0>], dictionary: [{couch_file_fd,{{file_descriptor,prim_file,{#Port<0.5682>,33}},"./data..."}},...], trap_exit: false, status: running, heap_size: 987, stack_size: 27, reductions: 5168
[error] 2019-12-05T08:37:28.639307Z nonode@nohost <0.1629.0> -------- CRASH REPORT Process (<0.1629.0>) with 2 neighbors crashed with reason: no match of right hand value {'EXIT',{{badmatch,eof},[{couch_file,read_raw_iolist_int,3,[{file,"src/couch_file.erl"},{line,627}]},{couch_file,handle_call,3,[{file,"src/couch_file.erl"},{line,449}]},{gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,615}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,647}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,247}]}]}} at couch_file:pread_binary/2(line:169) <= couch_file:pread_term/2(line:157) <= couch_btree:get_node/2(line:434) <= couch_btree:lookup/3(line:284) <= couch_btree:lookup/2(line:274) <= couch_bt_engine:open_local_docs/2(line:335) <= couch_db:open_doc_int/3(line:1622) <= couch_db:open_doc/3(line:281); initial_call: {couch_index,init,['Argument__1']}, ancestors: [<0.1628.0>], messages: [], links: [<0.1628.0>,<0.1630.0>], dictionary: [], trap_exit: false, status: running, heap_size: 2586, stack_size: 27, reductions: 1673
[error] 2019-12-05T08:37:28.639325Z nonode@nohost <0.257.0> -------- Error opening view group `_auth` from database `shards/a0000000-bfffffff/_users.1570022815`: {'EXIT',{{badmatch,{'EXIT',{{badmatch,eof},[{couch_file,read_raw_iolist_int,3,[{file,"src/couch_file.erl"},{line,627}]},{couch_file,handle_call,3,[{file,"src/couch_file.erl"},{line,449}]},{gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,615}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,647}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,247}]}]}}},{gen_server,call,[couch_index_server,{get_index,{couch_mrview_index,{mrst,<<62,130,60,42,67,131,172,12,24,212,229,116,19,90,91,8>>,nil,undefined,<<"shards/a0000000-bfffffff/_users.1570022815">>,<<"_design/_auth">>,<<"javascript">>,[],false,false,{[]},[],nil,nil,0,0,undefined,undefined,undefined,undefined,undefined,nil},<<"shards/a0000000-bfffffff/_users.1570022815">>,<<62,130,60,42,67,131,172,12,24,212,229,116,19,90,91,8>>}},infinity]}}}
[error] 2019-12-05T08:37:28.639653Z nonode@nohost <0.218.0> -------- gen_server couch_index_server terminated with reason: no match of right hand value {'EXIT',{{badmatch,eof},[{couch_file,read_raw_iolist_int,3,[{file,"src/couch_file.erl"},{line,627}]},{couch_file,handle_call,3,[{file,"src/couch_file.erl"},{line,449}]},{gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,615}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,647}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,247}]}]}} at couch_index_server:handle_info/2(line:190) <= gen_server:try_dispatch/4(line:601) <= gen_server:handle_msg/5(line:667) <= proc_lib:init_p_do_apply/3(line:247)
last msg: {'EXIT',<0.1628.0>,{badmatch,{'EXIT',{{badmatch,eof},[{couch_file,read_raw_iolist_int,3,[{file,"src/couch_file.erl"},{line,627}]},{couch_file,handle_call,3,[{file,"src/couch_file.erl"},{line,449}]},{gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,615}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,647}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,247}]}]}}}}
state: {st,"./data"}
extra: []
[error] 2019-12-05T08:37:28.639805Z nonode@nohost <0.218.0> -------- CRASH REPORT Process couch_index_server (<0.218.0>) with 1 neighbors exited with reason: no match of right hand value {'EXIT',{{badmatch,eof},[{couch_file,read_raw_iolist_int,3,[{file,"src/couch_file.erl"},{line,627}]},{couch_file,handle_call,3,[{file,"src/couch_file.erl"},{line,449}]},{gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,615}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,647}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,247}]}]}} at gen_server:terminate/7(line:812) <= proc_lib:init_p_do_apply/3(line:247); initial_call: {couch_index_server,init,['Argument__1']}, ancestors: [couch_secondary_services,couch_sup,<0.206.0>], messages: [], links: [<0.220.0>,<0.216.0>], dictionary: [], trap_exit: true, status: running, heap_size: 1598, stack_size: 27, reductions: 882
[error] 2019-12-05T08:37:28.639982Z nonode@nohost <0.216.0> -------- Supervisor couch_secondary_services had child index_server started with couch_index_server:start_link() at <0.218.0> exit with reason {badmatch,{'EXIT',{{badmatch,eof},[{couch_file,read_raw_iolist_int,3,[{file,"src/couch_file.erl"},{line,627}]},{couch_file,handle_call,3,[{file,"src/couch_file.erl"},{line,449}]},{gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,615}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,647}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,247}]}]}}} in context child_terminated
```
## Expected Behavior
Startup of couchdb
## Current Behavior
couchdb unaccessible/crashed
## Possible Solution
/
## Steps to Reproduce (for bugs)
Dockerfile is
```
FROM couchdb:2.3.1
COPY _docker/couchdb/local.ini /opt/couchdb/etc/local.d/my.ini
COPY _docker/couchdb/couchdbinit.sh /opt/couchdbinit.sh
RUN chmod +x /opt/couchdbinit.sh
```
my.ini is
```
[httpd]
enable_cors = true
[cors]
origins = *
credentials = true
; List of accepted methods, comma-separated
; refer to http://tools.ietf.org/html/rfc2616, rfc2817, rfc5789
methods = GET, POST, PUT, DELETE
headers = accept, authorization, content-type, origin, referer, x-csrf-token
```
>
> the couchdbinit.sh is just used to create a db and some users on boot
>
>
## Your Environment
* Version used: 2.3.1
* Browser Name and version: Chrome 78
* Operating System and version (desktop or mobile): Window 10
@wohali says: Looks like the db file got truncated by NFS and there may be data loss, but I'd appreciate it if someone else could take a look. What happens if you restore your database from backup?
Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services