all_dbs_active error and continuous replications

[Gregor Martynus <gr...@martynus.net>]

Hey everybody,

I've an app where an own db gets created for each user which gets
replicated continuously to a big central db.
This works great, it gives me fantastic possibilities that wouldn't be
possible with other databases, I love it.

My problem is though, that at some point I run into the all_dbs_active
error. Once I got there, everything freezes. I have to restart couchDB,
that's really bad. From what I can tell, continuous replications to count
against this limit, is that correct? Does that mean, that every continuous
replication keeps the db file open, even if there is no change for a longer
time?

My big question is: is there anything I can do about it? I mean to make
this setup scale better? I know I can increase the limit in configuration,
but that would just give me some more time, not solve my problem.

I'm really concerned ... and I asked around everywhere, but nobody could
give me an answer yet. Please help, I'd really appreciate it.

Thanks!

× Gregor

Re: all_dbs_active error and continuous replications

["Johannes J. Schmidt" <sc...@netzmerk.com>]

Thanks Gregor for raising this question. I have a very similar setup and
I am also concerned about how it will scale.

g jo
Am Dienstag, den 06.12.2011, 10:29 +0100 schrieb Gregor Martynus:
> Thanks Benoit!
> 
> I'm sorry to keep bugging ...
> 
> The problem is, at least for me, that I can't simply delete docs from the
> _replicator db. It's a critical part of my couchDB setup. It is something
> that excited me a lot a about couchDB, I just want to setup these
> replications and then don't think about it any more, that's something that
> the DB should just do – and let me relax.
> 
> What I really don't understand is, _why_ the continuous replication keep
> the db files open? I'm not an Erlang developer at all, sorry if this is a
> dump question/thought, but shouldn't the continuous replication be
> triggered only if a change happens in the db, instead of keeping it alive
> all the time? Is this a bug or probably something that might be improved in
> future?
> 
> Thanks
> 
> × Gregor
> 
> On Tue, Dec 6, 2011 at 10:16 AM, Benoit Chesneau <bc...@gmail.com>wrote:
> 
> > On Tue, Dec 6, 2011 at 10:11 AM, Gregor Martynus <gr...@martynus.net>
> > wrote:
> > > Thanks Benoit, I know about this setting. But at some point, this won't
> > > work anymore. My question is if there is any way to stop continuous
> > > replications to keep the db files active changes have been replicated?
> > Does
> > > that make sense?
> > >
> > >
> >
> > other way would be to queue changes events and when replicated dbs <
> > max dbs open a new task if a change happened on this . Something like
> > it.
> >
> > Otherwise to answer to your question, if you use the _replicator db
> > you can interrupt a task by just deleting the doc.
> >
> > - benoît
> >

Re: all_dbs_active error and continuous replications

[Filipe David Manana <fd...@apache.org>]

On Tue, Dec 6, 2011 at 9:29 AM, Gregor Martynus <gr...@martynus.net> wrote:
> What I really don't understand is, _why_ the continuous replication keep
> the db files open? I'm not an Erlang developer at all, sorry if this is a
> dump question/thought, but shouldn't the continuous replication be
> triggered only if a change happens in the db, instead of keeping it alive
> all the time? Is this a bug or probably something that might be improved in
> future?

Actually it doesn't keep the source and database open when there's no
changes to transfer from the source to the target.
What happens is that when started, the replication records the
databases' instance start times - they're timestamps that tell when
the databases where open by the couchdb server.

Whenever the replication commits a checkpoint (it is committed against
the source and the target), it verifies if the endpoints' current
instance start times are the same as when it started. If they're not
the same, then you'll get that max_dbs_open error message from the
replicator.

When the system has max_dbs_open and a request to open another
database X comes, it will close the least recently used database Y in
order to open the new one. If later on a request to open the database
X that was previously closed (because we reached max_dbs_open)
arrives, the new least recently used database will be closed, and X
will be reopened - at this point it gets a new instance start
timestamp.

The reason to abort if the instance start timestamps changed, it to
prevent you from getting into an inconsistent state. For example,
during an idle period (when there's no changes to transfer from source
to target) someone could swap database files - if the replicator
ignored the timestamps, it could mean in the end your target would be
missing all the changes transferred before the file swap.

One thing I have been thinking for sometime is to have the server
remember the timestamps and header md5 of each database opened since
it started. When it reopens a database that was previously closed by
the LRU system (due to max_dbs_open), it would use the cached
timestamp if the database's header md5 is the same as the one that's
cached along with the timestamp.
I haven't convinced myself this is a flawless/good idea however and
never made extensive tests.


>
> Thanks
>
> × Gregor
>
> On Tue, Dec 6, 2011 at 10:16 AM, Benoit Chesneau <bc...@gmail.com>wrote:
>
>> On Tue, Dec 6, 2011 at 10:11 AM, Gregor Martynus <gr...@martynus.net>
>> wrote:
>> > Thanks Benoit, I know about this setting. But at some point, this won't
>> > work anymore. My question is if there is any way to stop continuous
>> > replications to keep the db files active changes have been replicated?
>> Does
>> > that make sense?
>> >
>> >
>>
>> other way would be to queue changes events and when replicated dbs <
>> max dbs open a new task if a change happened on this . Something like
>> it.
>>
>> Otherwise to answer to your question, if you use the _replicator db
>> you can interrupt a task by just deleting the doc.
>>
>> - benoît
>>



-- 
Filipe David Manana,

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."

Re: all_dbs_active error and continuous replications

[Gregor Martynus <gr...@martynus.net>]

Thanks Benoit!

I'm sorry to keep bugging ...

The problem is, at least for me, that I can't simply delete docs from the
_replicator db. It's a critical part of my couchDB setup. It is something
that excited me a lot a about couchDB, I just want to setup these
replications and then don't think about it any more, that's something that
the DB should just do – and let me relax.

What I really don't understand is, _why_ the continuous replication keep
the db files open? I'm not an Erlang developer at all, sorry if this is a
dump question/thought, but shouldn't the continuous replication be
triggered only if a change happens in the db, instead of keeping it alive
all the time? Is this a bug or probably something that might be improved in
future?

Thanks

× Gregor

On Tue, Dec 6, 2011 at 10:16 AM, Benoit Chesneau <bc...@gmail.com>wrote:

> On Tue, Dec 6, 2011 at 10:11 AM, Gregor Martynus <gr...@martynus.net>
> wrote:
> > Thanks Benoit, I know about this setting. But at some point, this won't
> > work anymore. My question is if there is any way to stop continuous
> > replications to keep the db files active changes have been replicated?
> Does
> > that make sense?
> >
> >
>
> other way would be to queue changes events and when replicated dbs <
> max dbs open a new task if a change happened on this . Something like
> it.
>
> Otherwise to answer to your question, if you use the _replicator db
> you can interrupt a task by just deleting the doc.
>
> - benoît
>

Re: all_dbs_active error and continuous replications

[Benoit Chesneau <bc...@gmail.com>]

On Tue, Dec 6, 2011 at 10:11 AM, Gregor Martynus <gr...@martynus.net> wrote:
> Thanks Benoit, I know about this setting. But at some point, this won't
> work anymore. My question is if there is any way to stop continuous
> replications to keep the db files active changes have been replicated? Does
> that make sense?
>
>

other way would be to queue changes events and when replicated dbs <
max dbs open a new task if a change happened on this . Something like
it.

Otherwise to answer to your question, if you use the _replicator db
you can interrupt a task by just deleting the doc.

- benoît

Re: all_dbs_active error and continuous replications

[Gregor Martynus <gr...@martynus.net>]

Thanks Benoit, I know about this setting. But at some point, this won't
work anymore. My question is if there is any way to stop continuous
replications to keep the db files active changes have been replicated? Does
that make sense?


On Tue, Dec 6, 2011 at 10:08 AM, Benoit Chesneau <bc...@gmail.com>wrote:

> On Tue, Dec 6, 2011 at 9:56 AM, Gregor Martynus <gr...@martynus.net>
> wrote:
> > Hey everybody,
> >
> > I've an app where an own db gets created for each user which gets
> > replicated continuously to a big central db.
> > This works great, it gives me fantastic possibilities that wouldn't be
> > possible with other databases, I love it.
> >
> > My problem is though, that at some point I run into the all_dbs_active
> > error. Once I got there, everything freezes. I have to restart couchDB,
> > that's really bad. From what I can tell, continuous replications to count
> > against this limit, is that correct? Does that mean, that every
> continuous
> > replication keeps the db file open, even if there is no change for a
> longer
> > time?
> >
> > My big question is: is there anything I can do about it? I mean to make
> > this setup scale better? I know I can increase the limit in
> configuration,
> > but that would just give me some more time, not solve my problem.
> >
> > I'm really concerned ... and I asked around everywhere, but nobody could
> > give me an answer yet. Please help, I'd really appreciate it.
> >
> > Thanks!
> >
> > × Gregor
>
> You can increase max_dbs_open in [couchdb] section of the ini. It's by
> default 100.
>
> - benoît
>

Re: all_dbs_active error and continuous replications

[Benoit Chesneau <bc...@gmail.com>]

On Tue, Dec 6, 2011 at 9:56 AM, Gregor Martynus <gr...@martynus.net> wrote:
> Hey everybody,
>
> I've an app where an own db gets created for each user which gets
> replicated continuously to a big central db.
> This works great, it gives me fantastic possibilities that wouldn't be
> possible with other databases, I love it.
>
> My problem is though, that at some point I run into the all_dbs_active
> error. Once I got there, everything freezes. I have to restart couchDB,
> that's really bad. From what I can tell, continuous replications to count
> against this limit, is that correct? Does that mean, that every continuous
> replication keeps the db file open, even if there is no change for a longer
> time?
>
> My big question is: is there anything I can do about it? I mean to make
> this setup scale better? I know I can increase the limit in configuration,
> but that would just give me some more time, not solve my problem.
>
> I'm really concerned ... and I asked around everywhere, but nobody could
> give me an answer yet. Please help, I'd really appreciate it.
>
> Thanks!
>
> × Gregor

You can increase max_dbs_open in [couchdb] section of the ini. It's by
default 100.

- benoît