You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Spike Grobstein <sp...@ticketevolution.com> on 2012/11/26 17:56:12 UTC
running out of space
Hi CouchDB mailing list,
I've been running Chef Server since about March and continue to run out of space. I've emailed the Chef mailing list and although they were able to help, none of their answers successfully solved my problem.
I'm running CouchDB v1.0.1:
chef-server:~$ curl localhost:5984
{"couchdb":"Welcome","version":"1.0.1"}
And I have a script running daily via cron that compacts the data and the views:
https://gist.github.com/4091940
unfortunately, this has not halted the growth of used space on my couchdb partition. It doesn't seem to be cleaning up deleted documents:
chef-server:~$ curl -H "Content-Type: application/json" localhost:5984/chef
{"db_name":"chef","doc_count":1235,"doc_del_count":10212,"update_seq":536514,"purge_seq":0,"compact_running":false,"disk_size":45922017384,"instance_start_time":"1353948344088704","disk_format_version":5,"committed_update_seq":536514}
Currently, the couchdb partition is using up 43GB of the 60GB allocated to it. Is there anything I can do?
The server is Ubuntu 12.04.1 LTS with all of the latest packages. I've tried rebooting the server and restarting the couchdb daemon (as suggested on the chef mailing list), but to no avail.
Any ideas?
Thanks!
...spike
Re: running out of space
Posted by Spike Grobstein <sp...@ticketevolution.com>.
You are 100% on the mark.
I just deleted that file (it was empty) and ran the compaction script again and after restarting couchdb, my database size went from over 40GB down to 11MB.
You guys rule! Thanks!
...spike
On Nov 26, 2012, at 12:16 PM, Adam Kocoloski wrote:
> You probably have an empty .compact file for the Chef DB. Remove that file and try again. We fixed this bug in a newer CouchDB release (can't remember which one off the top of my head). Cheers,
>
> Adam
>
> On Nov 26, 2012, at 12:11 PM, Spike Grobstein <sp...@ticketevolution.com> wrote:
>
>> Hi Robert,
>>
>> Wow. You are correct. I can't believe I never thought to look at the logs. I guess when the HTTP requests never failed, I didn't think to look there.
>>
>> https://gist.github.com/4149393
>>
>> It's loading up with errors related to
>>
>> [Mon, 26 Nov 2012 17:06:54 GMT] [error] [emulator] Error in process <0.20250.6> with exit value: {{badmatch,no_valid_header},[{couch_db_updater,start_copy_compact,1}]}
>>
>> I have almost no experience with couch administration outside of what I've done with chef.
>>
>> What could be the reason for this?
>>
>> Thanks!
>>
>>
>> ...spike
>>
>>
>> On Nov 26, 2012, at 12:01 PM, Robert Newson wrote:
>>
>>> Is the compaction completing successfully? The log should indicate that.
>>>
>>>
>>> On 26 November 2012 16:56, Spike Grobstein <sp...@ticketevolution.com>wrote:
>>>
>>>> Hi CouchDB mailing list,
>>>>
>>>> I've been running Chef Server since about March and continue to run out of
>>>> space. I've emailed the Chef mailing list and although they were able to
>>>> help, none of their answers successfully solved my problem.
>>>>
>>>> I'm running CouchDB v1.0.1:
>>>>
>>>> chef-server:~$ curl localhost:5984
>>>> {"couchdb":"Welcome","version":"1.0.1"}
>>>>
>>>> And I have a script running daily via cron that compacts the data and the
>>>> views:
>>>>
>>>> https://gist.github.com/4091940
>>>>
>>>> unfortunately, this has not halted the growth of used space on my couchdb
>>>> partition. It doesn't seem to be cleaning up deleted documents:
>>>>
>>>> chef-server:~$ curl -H "Content-Type: application/json" localhost:5984/chef
>>>>
>>>> {"db_name":"chef","doc_count":1235,"doc_del_count":10212,"update_seq":536514,"purge_seq":0,"compact_running":false,"disk_size":45922017384,"instance_start_time":"1353948344088704","disk_format_version":5,"committed_update_seq":536514}
>>>>
>>>> Currently, the couchdb partition is using up 43GB of the 60GB allocated to
>>>> it. Is there anything I can do?
>>>>
>>>> The server is Ubuntu 12.04.1 LTS with all of the latest packages. I've
>>>> tried rebooting the server and restarting the couchdb daemon (as suggested
>>>> on the chef mailing list), but to no avail.
>>>>
>>>> Any ideas?
>>>>
>>>> Thanks!
>>>>
>>>>
>>>> ...spike
>>
>
Re: running out of space
Posted by Adam Kocoloski <ko...@apache.org>.
You probably have an empty .compact file for the Chef DB. Remove that file and try again. We fixed this bug in a newer CouchDB release (can't remember which one off the top of my head). Cheers,
Adam
On Nov 26, 2012, at 12:11 PM, Spike Grobstein <sp...@ticketevolution.com> wrote:
> Hi Robert,
>
> Wow. You are correct. I can't believe I never thought to look at the logs. I guess when the HTTP requests never failed, I didn't think to look there.
>
> https://gist.github.com/4149393
>
> It's loading up with errors related to
>
> [Mon, 26 Nov 2012 17:06:54 GMT] [error] [emulator] Error in process <0.20250.6> with exit value: {{badmatch,no_valid_header},[{couch_db_updater,start_copy_compact,1}]}
>
> I have almost no experience with couch administration outside of what I've done with chef.
>
> What could be the reason for this?
>
> Thanks!
>
>
> ...spike
>
>
> On Nov 26, 2012, at 12:01 PM, Robert Newson wrote:
>
>> Is the compaction completing successfully? The log should indicate that.
>>
>>
>> On 26 November 2012 16:56, Spike Grobstein <sp...@ticketevolution.com>wrote:
>>
>>> Hi CouchDB mailing list,
>>>
>>> I've been running Chef Server since about March and continue to run out of
>>> space. I've emailed the Chef mailing list and although they were able to
>>> help, none of their answers successfully solved my problem.
>>>
>>> I'm running CouchDB v1.0.1:
>>>
>>> chef-server:~$ curl localhost:5984
>>> {"couchdb":"Welcome","version":"1.0.1"}
>>>
>>> And I have a script running daily via cron that compacts the data and the
>>> views:
>>>
>>> https://gist.github.com/4091940
>>>
>>> unfortunately, this has not halted the growth of used space on my couchdb
>>> partition. It doesn't seem to be cleaning up deleted documents:
>>>
>>> chef-server:~$ curl -H "Content-Type: application/json" localhost:5984/chef
>>>
>>> {"db_name":"chef","doc_count":1235,"doc_del_count":10212,"update_seq":536514,"purge_seq":0,"compact_running":false,"disk_size":45922017384,"instance_start_time":"1353948344088704","disk_format_version":5,"committed_update_seq":536514}
>>>
>>> Currently, the couchdb partition is using up 43GB of the 60GB allocated to
>>> it. Is there anything I can do?
>>>
>>> The server is Ubuntu 12.04.1 LTS with all of the latest packages. I've
>>> tried rebooting the server and restarting the couchdb daemon (as suggested
>>> on the chef mailing list), but to no avail.
>>>
>>> Any ideas?
>>>
>>> Thanks!
>>>
>>>
>>> ...spike
>
Re: running out of space
Posted by Robert Newson <rn...@apache.org>.
Do you have a chef.couch.compact file on disk?
On 26 November 2012 17:11, Spike Grobstein <sp...@ticketevolution.com>wrote:
> Hi Robert,
>
> Wow. You are correct. I can't believe I never thought to look at the logs.
> I guess when the HTTP requests never failed, I didn't think to look there.
>
> https://gist.github.com/4149393
>
> It's loading up with errors related to
>
> [Mon, 26 Nov 2012 17:06:54 GMT] [error] [emulator] Error in process
> <0.20250.6> with exit value:
> {{badmatch,no_valid_header},[{couch_db_updater,start_copy_compact,1}]}
>
> I have almost no experience with couch administration outside of what I've
> done with chef.
>
> What could be the reason for this?
>
> Thanks!
>
>
> ...spike
>
>
> On Nov 26, 2012, at 12:01 PM, Robert Newson wrote:
>
> > Is the compaction completing successfully? The log should indicate that.
> >
> >
> > On 26 November 2012 16:56, Spike Grobstein <spike@ticketevolution.com
> >wrote:
> >
> >> Hi CouchDB mailing list,
> >>
> >> I've been running Chef Server since about March and continue to run out
> of
> >> space. I've emailed the Chef mailing list and although they were able to
> >> help, none of their answers successfully solved my problem.
> >>
> >> I'm running CouchDB v1.0.1:
> >>
> >> chef-server:~$ curl localhost:5984
> >> {"couchdb":"Welcome","version":"1.0.1"}
> >>
> >> And I have a script running daily via cron that compacts the data and
> the
> >> views:
> >>
> >> https://gist.github.com/4091940
> >>
> >> unfortunately, this has not halted the growth of used space on my
> couchdb
> >> partition. It doesn't seem to be cleaning up deleted documents:
> >>
> >> chef-server:~$ curl -H "Content-Type: application/json"
> localhost:5984/chef
> >>
> >>
> {"db_name":"chef","doc_count":1235,"doc_del_count":10212,"update_seq":536514,"purge_seq":0,"compact_running":false,"disk_size":45922017384,"instance_start_time":"1353948344088704","disk_format_version":5,"committed_update_seq":536514}
> >>
> >> Currently, the couchdb partition is using up 43GB of the 60GB allocated
> to
> >> it. Is there anything I can do?
> >>
> >> The server is Ubuntu 12.04.1 LTS with all of the latest packages. I've
> >> tried rebooting the server and restarting the couchdb daemon (as
> suggested
> >> on the chef mailing list), but to no avail.
> >>
> >> Any ideas?
> >>
> >> Thanks!
> >>
> >>
> >> ...spike
>
>
Re: running out of space
Posted by Spike Grobstein <sp...@ticketevolution.com>.
Hi Robert,
Wow. You are correct. I can't believe I never thought to look at the logs. I guess when the HTTP requests never failed, I didn't think to look there.
https://gist.github.com/4149393
It's loading up with errors related to
[Mon, 26 Nov 2012 17:06:54 GMT] [error] [emulator] Error in process <0.20250.6> with exit value: {{badmatch,no_valid_header},[{couch_db_updater,start_copy_compact,1}]}
I have almost no experience with couch administration outside of what I've done with chef.
What could be the reason for this?
Thanks!
...spike
On Nov 26, 2012, at 12:01 PM, Robert Newson wrote:
> Is the compaction completing successfully? The log should indicate that.
>
>
> On 26 November 2012 16:56, Spike Grobstein <sp...@ticketevolution.com>wrote:
>
>> Hi CouchDB mailing list,
>>
>> I've been running Chef Server since about March and continue to run out of
>> space. I've emailed the Chef mailing list and although they were able to
>> help, none of their answers successfully solved my problem.
>>
>> I'm running CouchDB v1.0.1:
>>
>> chef-server:~$ curl localhost:5984
>> {"couchdb":"Welcome","version":"1.0.1"}
>>
>> And I have a script running daily via cron that compacts the data and the
>> views:
>>
>> https://gist.github.com/4091940
>>
>> unfortunately, this has not halted the growth of used space on my couchdb
>> partition. It doesn't seem to be cleaning up deleted documents:
>>
>> chef-server:~$ curl -H "Content-Type: application/json" localhost:5984/chef
>>
>> {"db_name":"chef","doc_count":1235,"doc_del_count":10212,"update_seq":536514,"purge_seq":0,"compact_running":false,"disk_size":45922017384,"instance_start_time":"1353948344088704","disk_format_version":5,"committed_update_seq":536514}
>>
>> Currently, the couchdb partition is using up 43GB of the 60GB allocated to
>> it. Is there anything I can do?
>>
>> The server is Ubuntu 12.04.1 LTS with all of the latest packages. I've
>> tried rebooting the server and restarting the couchdb daemon (as suggested
>> on the chef mailing list), but to no avail.
>>
>> Any ideas?
>>
>> Thanks!
>>
>>
>> ...spike
Re: running out of space
Posted by Robert Newson <rn...@apache.org>.
Is the compaction completing successfully? The log should indicate that.
On 26 November 2012 16:56, Spike Grobstein <sp...@ticketevolution.com>wrote:
> Hi CouchDB mailing list,
>
> I've been running Chef Server since about March and continue to run out of
> space. I've emailed the Chef mailing list and although they were able to
> help, none of their answers successfully solved my problem.
>
> I'm running CouchDB v1.0.1:
>
> chef-server:~$ curl localhost:5984
> {"couchdb":"Welcome","version":"1.0.1"}
>
> And I have a script running daily via cron that compacts the data and the
> views:
>
> https://gist.github.com/4091940
>
> unfortunately, this has not halted the growth of used space on my couchdb
> partition. It doesn't seem to be cleaning up deleted documents:
>
> chef-server:~$ curl -H "Content-Type: application/json" localhost:5984/chef
>
> {"db_name":"chef","doc_count":1235,"doc_del_count":10212,"update_seq":536514,"purge_seq":0,"compact_running":false,"disk_size":45922017384,"instance_start_time":"1353948344088704","disk_format_version":5,"committed_update_seq":536514}
>
> Currently, the couchdb partition is using up 43GB of the 60GB allocated to
> it. Is there anything I can do?
>
> The server is Ubuntu 12.04.1 LTS with all of the latest packages. I've
> tried rebooting the server and restarting the couchdb daemon (as suggested
> on the chef mailing list), but to no avail.
>
> Any ideas?
>
> Thanks!
>
>
> ...spike