You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by f z <fu...@gmail.com> on 2021/03/26 13:17:08 UTC

problem with disappearing db

I just had a strange problem with an installation of couchdb.

After restarting the couchdb service, it seems it can't access anymore to
some of the dbs that were there before.
The files of those dbs seems to be still present in the shards folder, but
after the restart, it seems the system can't access them.
Just before the restart, the dbs were there and working regularly, there
was no delete operation.
The couchdb configuration, at least to my knowledge, did not change since
installation, and by my check seems to be the standard one.
Couchdb is version 2.3.1.

I've also tried on a different machine copying one of our previous
timed-backups (we do a daily snapshot of the whole couch db folders), but
once restored seems to have the exact same problem: the couchdb service
comes up, but the "documents" and "main" dbs are missing.

I've no idea were to start debugging this, and if there is a way of
recovering those dbs from the backup I've right now.

Anyway, the logs during the restart are the following:

[info] 2021-03-26T09:06:57.909241Z couchdb@127.0.0.1 <0.56.0> --------
alarm_handler: {set,{system_memory_high_watermark,[]}}
[error] 2021-03-26T09:10:26.491449Z couchdb@127.0.0.1 <0.64.0> --------
gen_server memsup terminated with reason: {port_died,normal}
  last msg: {'EXIT',<0.65.0>,{port_died,normal}}
     state: [{data,[{"Timeout",60000}]},{items,{"Memory
Usage",[{"Allocated",7946051584},{"Total",8171896832}]}},{items,{"Worst
Memory User",[{"Pid",<0.218.0>},{"Memory",7687816}]}}]
    extra: []
[info] 2021-03-26T09:10:32.894224Z couchdb@127.0.0.1 <0.9.0> --------
Application couch_log started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:32.899154Z couchdb@127.0.0.1 <0.9.0> --------
Application folsom started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.012320Z couchdb@127.0.0.1 <0.9.0> --------
Application couch_stats started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.012544Z couchdb@127.0.0.1 <0.9.0> --------
Application khash started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.026786Z couchdb@127.0.0.1 <0.9.0> --------
Application couch_event started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.027019Z couchdb@127.0.0.1 <0.9.0> --------
Application hyper started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.038401Z couchdb@127.0.0.1 <0.9.0> --------
Application ibrowse started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.048224Z couchdb@127.0.0.1 <0.9.0> --------
Application ioq started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.048468Z couchdb@127.0.0.1 <0.9.0> --------
Application mochiweb started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.064719Z couchdb@127.0.0.1 <0.212.0> --------
Apache CouchDB 2.3.1 is starting.

[info] 2021-03-26T09:10:33.064980Z couchdb@127.0.0.1 <0.213.0> --------
Starting couch_sup
[notice] 2021-03-26T09:10:33.082657Z couchdb@127.0.0.1 <0.96.0> --------
config: [features] pluggable-storage-engines set to true for reason nil
[notice] 2021-03-26T09:10:33.097461Z couchdb@127.0.0.1 <0.96.0> --------
config: [admins] admin set to XXX for reason nil
[info] 2021-03-26T09:10:33.258955Z couchdb@127.0.0.1 <0.212.0> --------
Apache CouchDB has started. Time to relax.

[info] 2021-03-26T09:10:33.259081Z couchdb@127.0.0.1 <0.212.0> --------
Apache CouchDB has started on http://127.0.0.1:5986/
[info] 2021-03-26T09:10:33.259276Z couchdb@127.0.0.1 <0.9.0> --------
Application couch started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.259725Z couchdb@127.0.0.1 <0.9.0> --------
Application ets_lru started on node 'couchdb@127.0.0.1'
[notice] 2021-03-26T09:10:33.290376Z couchdb@127.0.0.1 <0.272.0> --------
rexi_server : started servers
[notice] 2021-03-26T09:10:33.295464Z couchdb@127.0.0.1 <0.279.0> --------
rexi_buffer : started servers
[info] 2021-03-26T09:10:33.295971Z couchdb@127.0.0.1 <0.9.0> --------
Application rexi started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.342338Z couchdb@127.0.0.1 <0.9.0> --------
Application mem3 started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.342560Z couchdb@127.0.0.1 <0.9.0> --------
Application fabric started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.403837Z couchdb@127.0.0.1 <0.9.0> --------
Application chttpd started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.436538Z couchdb@127.0.0.1 <0.9.0> --------
Application couch_index started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.437227Z couchdb@127.0.0.1 <0.9.0> --------
Application couch_mrview started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.437531Z couchdb@127.0.0.1 <0.9.0> --------
Application couch_plugins started on node 'couchdb@127.0.0.1'
[notice] 2021-03-26T09:10:33.559220Z couchdb@127.0.0.1 <0.96.0> --------
config: [features] scheduler set to true for reason nil
[info] 2021-03-26T09:10:33.620740Z couchdb@127.0.0.1 <0.9.0> --------
Application couch_replicator started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.649361Z couchdb@127.0.0.1 <0.9.0> --------
Application couch_peruser started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.703726Z couchdb@127.0.0.1 <0.9.0> --------
Application ddoc_cache started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.734466Z couchdb@127.0.0.1 <0.9.0> --------
Application global_changes started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.734529Z couchdb@127.0.0.1 <0.9.0> --------
Application jiffy started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.755574Z couchdb@127.0.0.1 <0.9.0> --------
Application mango started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.777359Z couchdb@127.0.0.1 <0.9.0> --------
Application setup started on node 'couchdb@127.0.0.1'
[info] 2021-03-26T09:10:33.777609Z couchdb@127.0.0.1 <0.9.0> --------
Application snappy started on node 'couchdb@127.0.0.1'
[notice] 2021-03-26T09:10:38.469290Z couchdb@127.0.0.1 <0.353.0> --------
couch_replicator_clustering : cluster stable
[notice] 2021-03-26T09:10:38.478642Z couchdb@127.0.0.1 <0.405.0> --------
Started replicator db changes listener <0.511.0>
[error] 2021-03-26T09:10:45.348964Z couchdb@127.0.0.1 emulator --------
Error in process <0.670.0> on node 'couchdb@127.0.0.1' with exit value:
{database_does_not_exist,[{mem3_shards,load_shards_from_db,"documents",[{file,"src/mem3_shards.erl"},{line,395}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,370}]},{mem3_shards,for_db,2,[{file,"src/mem3_shards.erl"},{line,54}]},{fabric_view_all_docs,go,5,[{file,"src/fabric_view_all_docs.erl"},{line,24}]},{couch_db,'-get_design_docs/1-fun-0-',1,[{file,"src/couch_db.erl"},{line,627}]}]}

[error] 2021-03-26T09:10:45.349035Z couchdb@127.0.0.1 <0.260.0> --------
Could not get design docs for
<<"shards/20000000-3fffffff/documents.1594915751">>
error:{database_does_not_exist,[{mem3_shards,load_shards_from_db,"documents",[{file,"src/mem3_shards.erl"},{line,395}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,370}]},{mem3_shards,for_db,2,[{file,"src/mem3_shards.erl"},{line,54}]},{fabric_view_all_docs,go,5,[{file,"src/fabric_view_all_docs.erl"},{line,24}]},{couch_db,'-get_design_docs/1-fun-0-',1,[{file,"src/couch_db.erl"},{line,627}]}]}
[notice] 2021-03-26T09:10:48.291445Z couchdb@127.0.0.1 <0.279.0> --------
rexi_buffer : cluster stable
[notice] 2021-03-26T09:10:48.291610Z couchdb@127.0.0.1 <0.272.0> --------
rexi_server : cluster stable
[error] 2021-03-26T09:11:06.416502Z couchdb@127.0.0.1 <0.260.0> --------
Could not get design docs for
<<"shards/20000000-3fffffff/main.1594915751">>
error:{database_does_not_exist,[{mem3_shards,load_shards_from_db,"main",[{file,"src/mem3_shards.erl"},{line,395}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,370}]},{mem3_shards,for_db,2,[{file,"src/mem3_shards.erl"},{line,54}]},{fabric_view_all_docs,go,5,[{file,"src/fabric_view_all_docs.erl"},{line,24}]},{couch_db,'-get_design_docs/1-fun-0-',1,[{file,"src/couch_db.erl"},{line,627}]}]}
[error] 2021-03-26T09:11:06.416806Z couchdb@127.0.0.1 emulator --------
Error in process <0.1116.0> on node 'couchdb@127.0.0.1' with exit value:
{database_does_not_exist,[{mem3_shards,load_shards_from_db,"main",[{file,"src/mem3_shards.erl"},{line,395}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,370}]},{mem3_shards,for_db,2,[{file,"src/mem3_shards.erl"},{line,54}]},{fabric_view_all_docs,go,5,[{file,"src/fabric_view_all_docs.erl"},{line,24}]},{couch_db,'-get_design_docs/1-fun-0-',1,[{file,"src/couch_db.erl"},{line,627}]}]}