You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Harry Vangberg <ha...@vangberg.name> on 2011/02/12 19:39:59 UTC
Replication causes huge drop in performance
I have a Couchdb 1.0.2 server running on an EC2 m1.xlarge (15GB / 4
cores) instance, with ~40m docs clocking in at ~45G. The database
files are stored on EBS, configured as a 250G RAID-10 split on 4 EBS
disks. I am trying to build a new slave instance on another EC2
instance. On the master I have max_http_pipeline_size=100 and
max_http_sessions=1. When I start a pull replication from the slave
instance, view requests to the master that usually have a ~1s response
time, jumps to a ~30-40s response time, which breaks my app, that
still relies on the master instance. It seems a bit aggressive. Any
ideas?
--
Harry Vangberg <ha...@vangberg.name> http://harry.vangberg.name
Re: Replication causes huge drop in performance
Posted by Harry Vangberg <ha...@vangberg.name>.
It should be mentioned that the replication is starting from scratch.
On 12 February 2011 15:39, Harry Vangberg <ha...@vangberg.name> wrote:
> I have a Couchdb 1.0.2 server running on an EC2 m1.xlarge (15GB / 4
> cores) instance, with ~40m docs clocking in at ~45G. The database
> files are stored on EBS, configured as a 250G RAID-10 split on 4 EBS
> disks. I am trying to build a new slave instance on another EC2
> instance. On the master I have max_http_pipeline_size=100 and
> max_http_sessions=1. When I start a pull replication from the slave
> instance, view requests to the master that usually have a ~1s response
> time, jumps to a ~30-40s response time, which breaks my app, that
> still relies on the master instance. It seems a bit aggressive. Any
> ideas?
>
> --
> Harry Vangberg <ha...@vangberg.name> http://harry.vangberg.name
>
--
Harry Vangberg <ha...@vangberg.name> http://harry.vangberg.name
Re: Replication causes huge drop in performance
Posted by Harry Vangberg <ha...@vangberg.name>.
Setting the session and pipeline size on the target did indeed help.
Still some performance drop, but a way more reasonable amount.
Thanks,
On 12 February 2011 19:10, Filipe David Manana <fd...@apache.org> wrote:
> On Sat, Feb 12, 2011 at 10:39 AM, Harry Vangberg <ha...@vangberg.name> wrote:
>> I have a Couchdb 1.0.2 server running on an EC2 m1.xlarge (15GB / 4
>> cores) instance, with ~40m docs clocking in at ~45G. The database
>> files are stored on EBS, configured as a 250G RAID-10 split on 4 EBS
>> disks. I am trying to build a new slave instance on another EC2
>> instance. On the master I have max_http_pipeline_size=100 and
>> max_http_sessions=1. When I start a pull replication from the slave
>> instance, view requests to the master that usually have a ~1s response
>> time, jumps to a ~30-40s response time, which breaks my app, that
>> still relies on the master instance. It seems a bit aggressive. Any
>> ideas?
>
> If you want to decrease the number of http connections (and pipeline
> size) used by the pull replication triggered at the slave, you have to
> touch the max http sessions and pipeline size parameters in the slave,
> not in the master.
>
> Do you see this problem when you query with "?stale=ok" ?
> Also, how complex are your view's map and reduce functions?
>
>>
>> --
>> Harry Vangberg <ha...@vangberg.name> http://harry.vangberg.name
>>
>
>
>
> --
> Filipe David Manana,
> fdmanana@gmail.com, fdmanana@apache.org
>
> "Reasonable men adapt themselves to the world.
> Unreasonable men adapt the world to themselves.
> That's why all progress depends on unreasonable men."
>
--
Harry Vangberg <ha...@vangberg.name> http://harry.vangberg.name
Re: Replication causes huge drop in performance
Posted by Filipe David Manana <fd...@apache.org>.
On Sat, Feb 12, 2011 at 10:39 AM, Harry Vangberg <ha...@vangberg.name> wrote:
> I have a Couchdb 1.0.2 server running on an EC2 m1.xlarge (15GB / 4
> cores) instance, with ~40m docs clocking in at ~45G. The database
> files are stored on EBS, configured as a 250G RAID-10 split on 4 EBS
> disks. I am trying to build a new slave instance on another EC2
> instance. On the master I have max_http_pipeline_size=100 and
> max_http_sessions=1. When I start a pull replication from the slave
> instance, view requests to the master that usually have a ~1s response
> time, jumps to a ~30-40s response time, which breaks my app, that
> still relies on the master instance. It seems a bit aggressive. Any
> ideas?
If you want to decrease the number of http connections (and pipeline
size) used by the pull replication triggered at the slave, you have to
touch the max http sessions and pipeline size parameters in the slave,
not in the master.
Do you see this problem when you query with "?stale=ok" ?
Also, how complex are your view's map and reduce functions?
>
> --
> Harry Vangberg <ha...@vangberg.name> http://harry.vangberg.name
>
--
Filipe David Manana,
fdmanana@gmail.com, fdmanana@apache.org
"Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men."