You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Marco Monteiro <ma...@textovirtual.com> on 2010/11/08 13:45:57 UTC
seplication timeout problem
I'm trying to use CouchDB with Amazon's EC2 micro instances. These are very
slow.
There are no official numbers, but I think they are more or less equivalent
to Pentium 2 at 200 MHz.
I have 100 databases. There are 10 "master" databases and each has 10
"slave" databases.
Each of the 10 master databases has many documents (around 50,000); the
slaves start with no
documents.
I want to have continuous filtered seplication between each master and its
slaves. When I start
the continuous seplications, I get lots of timeouts in the seplication
processes. I think that part
of the problem is the communication with the JavaScript process for running
the filters. These
would go away if I implemented the filters in Erlang, right?
Anyway, I'm trying to tweak the configuration to prevent the timeouts. I'm
specifically playing with
os_process_timeout, which I understand, and with max_http_sessions,
max_http_pipeline_size,
which I don't understand. What do these do?
Anybody has any advice for me to get rid of the timeouts? Any other option
in the configuration
option that I can use?
Thanks,
Marco
Re: seplication timeout problem
Posted by Randall Leeds <ra...@gmail.com>.
On Mon, Nov 8, 2010 at 14:29, Marco Monteiro <ma...@textovirtual.com> wrote:
> First I get some "OS Process Error <0.526.0> :: {os_process_error,"OS
> process timed out."}"
> Then there are a lot of
>
> changes_loop died with reason {timeout,
> {gen_server,call,
> [couch_query_servers,
> {unlink_proc,<0.348.0>}]}}
>
> I have increased os_process_timeout to 60 seconds and that appears to reduce
> the number of timeouts.
>
That's the right move.
Re: seplication timeout problem
Posted by Marco Monteiro <ma...@textovirtual.com>.
First I get some "OS Process Error <0.526.0> :: {os_process_error,"OS
process timed out."}"
Then there are a lot of
changes_loop died with reason {timeout,
{gen_server,call,
[couch_query_servers,
{unlink_proc,<0.348.0>}]}}
I have increased os_process_timeout to 60 seconds and that appears to reduce
the number of timeouts.
As the machine has very limited resources, I am porting the filters to an
Erlang
design document. I am betting on this to help solve the problem.
Thanks,
Marco
On 8 November 2010 20:11, Randall Leeds <ra...@gmail.com> wrote:
> max_http_sessions and max_http_pipeline should control how many
> connections and how many pipeline requests the replicator is allowed
> to make per host.
> Right now this is actually broken for pull replication and the
> settings don't do anything, but the fix is (mostly) easy. On the other
> hand I think Filipe has been rewriting much of the replicator code so
> maybe it's not worth patching right now.
>
> If you tell us what timeout errors you're seeing and where I might be
> able to tell you how you could correct it.
> For example, if max_http_sessions and max_http_pipeline is the problem
> you would see "couch_rep_httpc request failed after 10 retries..." in
> the couch server log.
>
> -Randall
>
> On Mon, Nov 8, 2010 at 04:45, Marco Monteiro <ma...@textovirtual.com>
> wrote:
> > I'm trying to use CouchDB with Amazon's EC2 micro instances. These are
> very
> > slow.
> > There are no official numbers, but I think they are more or less
> equivalent
> > to Pentium 2 at 200 MHz.
> >
> > I have 100 databases. There are 10 "master" databases and each has 10
> > "slave" databases.
> > Each of the 10 master databases has many documents (around 50,000); the
> > slaves start with no
> > documents.
> >
> > I want to have continuous filtered seplication between each master and
> its
> > slaves. When I start
> > the continuous seplications, I get lots of timeouts in the seplication
> > processes. I think that part
> > of the problem is the communication with the JavaScript process for
> running
> > the filters. These
> > would go away if I implemented the filters in Erlang, right?
> >
> > Anyway, I'm trying to tweak the configuration to prevent the timeouts.
> I'm
> > specifically playing with
> > os_process_timeout, which I understand, and with max_http_sessions,
> > max_http_pipeline_size,
> > which I don't understand. What do these do?
> >
> > Anybody has any advice for me to get rid of the timeouts? Any other
> option
> > in the configuration
> > option that I can use?
> >
> > Thanks,
> > Marco
> >
>
Re: seplication timeout problem
Posted by Randall Leeds <ra...@gmail.com>.
max_http_sessions and max_http_pipeline should control how many
connections and how many pipeline requests the replicator is allowed
to make per host.
Right now this is actually broken for pull replication and the
settings don't do anything, but the fix is (mostly) easy. On the other
hand I think Filipe has been rewriting much of the replicator code so
maybe it's not worth patching right now.
If you tell us what timeout errors you're seeing and where I might be
able to tell you how you could correct it.
For example, if max_http_sessions and max_http_pipeline is the problem
you would see "couch_rep_httpc request failed after 10 retries..." in
the couch server log.
-Randall
On Mon, Nov 8, 2010 at 04:45, Marco Monteiro <ma...@textovirtual.com> wrote:
> I'm trying to use CouchDB with Amazon's EC2 micro instances. These are very
> slow.
> There are no official numbers, but I think they are more or less equivalent
> to Pentium 2 at 200 MHz.
>
> I have 100 databases. There are 10 "master" databases and each has 10
> "slave" databases.
> Each of the 10 master databases has many documents (around 50,000); the
> slaves start with no
> documents.
>
> I want to have continuous filtered seplication between each master and its
> slaves. When I start
> the continuous seplications, I get lots of timeouts in the seplication
> processes. I think that part
> of the problem is the communication with the JavaScript process for running
> the filters. These
> would go away if I implemented the filters in Erlang, right?
>
> Anyway, I'm trying to tweak the configuration to prevent the timeouts. I'm
> specifically playing with
> os_process_timeout, which I understand, and with max_http_sessions,
> max_http_pipeline_size,
> which I don't understand. What do these do?
>
> Anybody has any advice for me to get rid of the timeouts? Any other option
> in the configuration
> option that I can use?
>
> Thanks,
> Marco
>