You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Dan Santner <da...@me.com> on 2016/01/14 18:46:43 UTC
how to force flush a db?
Is there a way to force couch to flush everything to disk? right now I’m stopping the database to make that happen but would be really nice to be able to do that while keeping it running.
Maybe this isn’t even necessary?
Re: how to force flush a db?
Posted by Dan Santner <da...@me.com>.
Clemens, thanks for the support. Always good to hear when someone else has done it first!
Joan, thanks for the info. I’ll make sure we take the views before the data.
> On Jan 15, 2016, at 6:24 PM, Joan Touzet <wo...@apache.org> wrote:
>
> One thing to note is that your views and your DB may be in different
> states when you pull files off the HDD sequentially, if they're large.
> If your views are "newer" than the DB file you have, it could cause
> issue. I recommend copying the .views subdirectory FIRST, then copying
> the database file. This will ensure the views are always the same
> age as the DB or older, meaning only a minor amount of catch-up is
> required to update the view to current if at all.
>
> -Joan
>
> ----- Original Message -----
>> From: "Clemens Stolle" <cl...@fastmail.com>
>> To: user@couchdb.apache.org
>> Sent: Friday, January 15, 2016 3:52:49 AM
>> Subject: Re: how to force flush a db?
>>
>> Hey Dan,
>> I have an app in production and we handle backups the way you're
>> describing. Every night we push a zip archive to s3. Works great.
>>
>> ---
>> Clemens
>>
>>> Am 14.01.2016 um 21:44 schrieb Dan Santner <da...@me.com>:
>>>
>>> Thanks for the response.
>>>
>>> I should probably ask a different question.
>>>
>>> I’ve been using git to take snapshots of my databases in certain
>>> states. Basically i put all the data files under git and then
>>> when I have my data the way I want I can freeze it for later
>>> replay by committing it to a branch in a repo.
>>>
>>> It works really well for me. I’m thinking of the similar mechanism
>>> for backup. Basically I’ll zip the data directory and store it in
>>> S3 periodically as a backup. I was concerned though that the
>>> state of the files might not be restartable if I don’t shut down
>>> the couchdb process before snapshotting.
>>>
>>> Anyone else doing something like this?
>>>
>>>> On Jan 14, 2016, at 1:00 PM, Robert Samuel Newson
>>>> <rn...@apache.org> wrote:
>>>>
>>>> [couchdb]
>>>> delayed_commits=false
>>>>
>>>> It’s false by default from 2.0 onward too.
>>>>
>>>> When set to true, a timer calls fsync once per second. I’d argue
>>>> your forcing isn’t necessary in either case.
>>>>
>>>> B.
>>>>
>>>>> On 14 Jan 2016, at 17:46, Dan Santner <da...@me.com> wrote:
>>>>>
>>>>> Is there a way to force couch to flush everything to disk? right
>>>>> now I’m stopping the database to make that happen but would be
>>>>> really nice to be able to do that while keeping it running.
>>>>>
>>>>>
>>>>> Maybe this isn’t even necessary?
>>>
>>
Re: how to force flush a db?
Posted by Joan Touzet <wo...@apache.org>.
One thing to note is that your views and your DB may be in different
states when you pull files off the HDD sequentially, if they're large.
If your views are "newer" than the DB file you have, it could cause
issue. I recommend copying the .views subdirectory FIRST, then copying
the database file. This will ensure the views are always the same
age as the DB or older, meaning only a minor amount of catch-up is
required to update the view to current if at all.
-Joan
----- Original Message -----
> From: "Clemens Stolle" <cl...@fastmail.com>
> To: user@couchdb.apache.org
> Sent: Friday, January 15, 2016 3:52:49 AM
> Subject: Re: how to force flush a db?
>
> Hey Dan,
> I have an app in production and we handle backups the way you're
> describing. Every night we push a zip archive to s3. Works great.
>
> ---
> Clemens
>
> > Am 14.01.2016 um 21:44 schrieb Dan Santner <da...@me.com>:
> >
> > Thanks for the response.
> >
> > I should probably ask a different question.
> >
> > I’ve been using git to take snapshots of my databases in certain
> > states. Basically i put all the data files under git and then
> > when I have my data the way I want I can freeze it for later
> > replay by committing it to a branch in a repo.
> >
> > It works really well for me. I’m thinking of the similar mechanism
> > for backup. Basically I’ll zip the data directory and store it in
> > S3 periodically as a backup. I was concerned though that the
> > state of the files might not be restartable if I don’t shut down
> > the couchdb process before snapshotting.
> >
> > Anyone else doing something like this?
> >
> >> On Jan 14, 2016, at 1:00 PM, Robert Samuel Newson
> >> <rn...@apache.org> wrote:
> >>
> >> [couchdb]
> >> delayed_commits=false
> >>
> >> It’s false by default from 2.0 onward too.
> >>
> >> When set to true, a timer calls fsync once per second. I’d argue
> >> your forcing isn’t necessary in either case.
> >>
> >> B.
> >>
> >>> On 14 Jan 2016, at 17:46, Dan Santner <da...@me.com> wrote:
> >>>
> >>> Is there a way to force couch to flush everything to disk? right
> >>> now I’m stopping the database to make that happen but would be
> >>> really nice to be able to do that while keeping it running.
> >>>
> >>>
> >>> Maybe this isn’t even necessary?
> >
>
Re: how to force flush a db?
Posted by Clemens Stolle <cl...@fastmail.com>.
Hey Dan,
I have an app in production and we handle backups the way you're describing. Every night we push a zip archive to s3. Works great.
---
Clemens
> Am 14.01.2016 um 21:44 schrieb Dan Santner <da...@me.com>:
>
> Thanks for the response.
>
> I should probably ask a different question.
>
> I’ve been using git to take snapshots of my databases in certain states. Basically i put all the data files under git and then when I have my data the way I want I can freeze it for later replay by committing it to a branch in a repo.
>
> It works really well for me. I’m thinking of the similar mechanism for backup. Basically I’ll zip the data directory and store it in S3 periodically as a backup. I was concerned though that the state of the files might not be restartable if I don’t shut down the couchdb process before snapshotting.
>
> Anyone else doing something like this?
>
>> On Jan 14, 2016, at 1:00 PM, Robert Samuel Newson <rn...@apache.org> wrote:
>>
>> [couchdb]
>> delayed_commits=false
>>
>> It’s false by default from 2.0 onward too.
>>
>> When set to true, a timer calls fsync once per second. I’d argue your forcing isn’t necessary in either case.
>>
>> B.
>>
>>> On 14 Jan 2016, at 17:46, Dan Santner <da...@me.com> wrote:
>>>
>>> Is there a way to force couch to flush everything to disk? right now I’m stopping the database to make that happen but would be really nice to be able to do that while keeping it running.
>>>
>>>
>>> Maybe this isn’t even necessary?
>
Re: how to force flush a db?
Posted by Vishesh Mittal <vi...@platomedical.com>.
Hi
There is no need to shut down couchdb to make snapshots. Since the database
file always is in a consistent state you can take the snapshot of the file
any time without any issue.
We use periodic ebs snapshots of the live database, and it works great.
Warm regards
On 15 Jan 2016 04:44, "Dan Santner" <da...@me.com> wrote:
> Thanks for the response.
>
> I should probably ask a different question.
>
> I’ve been using git to take snapshots of my databases in certain states.
> Basically i put all the data files under git and then when I have my data
> the way I want I can freeze it for later replay by committing it to a
> branch in a repo.
>
> It works really well for me. I’m thinking of the similar mechanism for
> backup. Basically I’ll zip the data directory and store it in S3
> periodically as a backup. I was concerned though that the state of the
> files might not be restartable if I don’t shut down the couchdb process
> before snapshotting.
>
> Anyone else doing something like this?
>
> > On Jan 14, 2016, at 1:00 PM, Robert Samuel Newson <rn...@apache.org>
> wrote:
> >
> > [couchdb]
> > delayed_commits=false
> >
> > It’s false by default from 2.0 onward too.
> >
> > When set to true, a timer calls fsync once per second. I’d argue your
> forcing isn’t necessary in either case.
> >
> > B.
> >
> >> On 14 Jan 2016, at 17:46, Dan Santner <da...@me.com> wrote:
> >>
> >> Is there a way to force couch to flush everything to disk? right now
> I’m stopping the database to make that happen but would be really nice to
> be able to do that while keeping it running.
> >>
> >>
> >> Maybe this isn’t even necessary?
> >
>
>
Re: how to force flush a db?
Posted by Dan Santner <da...@me.com>.
Thanks for the response.
I should probably ask a different question.
I’ve been using git to take snapshots of my databases in certain states. Basically i put all the data files under git and then when I have my data the way I want I can freeze it for later replay by committing it to a branch in a repo.
It works really well for me. I’m thinking of the similar mechanism for backup. Basically I’ll zip the data directory and store it in S3 periodically as a backup. I was concerned though that the state of the files might not be restartable if I don’t shut down the couchdb process before snapshotting.
Anyone else doing something like this?
> On Jan 14, 2016, at 1:00 PM, Robert Samuel Newson <rn...@apache.org> wrote:
>
> [couchdb]
> delayed_commits=false
>
> It’s false by default from 2.0 onward too.
>
> When set to true, a timer calls fsync once per second. I’d argue your forcing isn’t necessary in either case.
>
> B.
>
>> On 14 Jan 2016, at 17:46, Dan Santner <da...@me.com> wrote:
>>
>> Is there a way to force couch to flush everything to disk? right now I’m stopping the database to make that happen but would be really nice to be able to do that while keeping it running.
>>
>>
>> Maybe this isn’t even necessary?
>
Re: how to force flush a db?
Posted by Robert Samuel Newson <rn...@apache.org>.
[couchdb]
delayed_commits=false
It’s false by default from 2.0 onward too.
When set to true, a timer calls fsync once per second. I’d argue your forcing isn’t necessary in either case.
B.
> On 14 Jan 2016, at 17:46, Dan Santner <da...@me.com> wrote:
>
> Is there a way to force couch to flush everything to disk? right now I’m stopping the database to make that happen but would be really nice to be able to do that while keeping it running.
>
>
> Maybe this isn’t even necessary?