You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Nick Vatamaniuc <va...@gmail.com> on 2022/09/01 05:01:24 UTC
Re: CPU constantly above 15% even when no client is talking to CouchDB
Hi Alfred,
It could be you're seeing the effect of busy waiting from Erlang VM
schedulers. Erlang VM would spawn a scheduler for each CPU core, and
then after any of those schedulers run out of work, they would still
spin in a busy loop for a bit waiting for more work. Sometimes that is
not desirable, especially in containerized environments so then it's
recommended to disable busy waiting by setting +sbwt to none in the
vm.args file. Give that a try and see if it helps. More info about it
can be found at https://adoptingerlang.org/docs/production/kubernetes/
Another reason could be that there are some background index building
or compaction jobs running.. Check out the output of _active_tasks to
see if that's the case.
Cheers,
-Nick
On Sun, Aug 14, 2022 at 11:38 AM Al Z. <az...@gmail.com> wrote:
>
> Hello.
> I have the latest CouchDB 3.2.2 on Ubuntu 22.04
>
> Even when there are no active clients talking to couchdb, the CPU is always
> above 15%.
> Not sure why.
> couchdb's beam.smp is currently the busiest process on the server.
>
> [image: image.png]
>
>
> # ps aux | grep couchdb
> couchdb 979 13.4 0.1 4647432 100284 ? Ssl 15:03 2:10
> /opt/couchdb/bin/../erts-12.2.1/bin/beam.smp -K true -A 16 -Bd -- -root
> /opt/couchdb/bin/.. -progname couchdb -- -home /opt/couchdb -- -boot
> /opt/couchdb/bin/../releases/3.2.2/couchdb -name couchdb@localhost
> -setcookie SOME_SECRET_COOKIE -kernel inet_dist_use_interface {127,0,0,1}
> -kernel error_logger silent -sasl sasl_error_logger false -noshell -noinput
> -smp enable -ssl session_lifetime 300 -- -config
> /opt/couchdb/bin/../releases/3.2.2/sys.config --
> couchdb 1111 0.0 0.0 7136 220 ? S 15:03 0:00
> /opt/couchdb/bin/../erts-12.2.1/bin/epmd -daemon
> couchdb 1163 0.0 0.0 2780 972 ? Ss 15:03 0:00
> erl_child_setup 65536
> couchdb 2727 0.0 0.0 3736 1192 ? Ss 15:04 0:00
> inet_gethost 4
> couchdb 2728 0.0 0.0 3736 108 ? S 15:04 0:00
> inet_gethost 4
> root 2982 0.0 0.0 6612 2220 pts/1 S+ 15:30 0:00 grep
> --color=auto couchdb
>
>
> Any hint will be appreciated.
>
> Thank you very much.
>
> Alfred.
Re: CPU constantly above 15% even when no client is talking to CouchDB
Posted by Nick Vatamaniuc <va...@gmail.com>.
Give `+sbwt none` a try and measure how it affects your cluster. It
might reduce the idle CPU usage a bit. The only adverse effect it
might have is it might increase the latency of requests, but it's
something you can measure.
The `+S ${SCHEDULERS}` is how you'd explicitly specify how many
scheduler threads to start, as you've read in the guide, it's mostly
when running in a containerized environment where the containers
"sees" all the CPUs and would start a scheduler for each one of them
even though its quota might effectively be only 1 CPUs.
There is also some background stats gathering and expiring of open
indices and db handles. Those won't show in active tasks. But without
access to the system or more details it's hard to recommend changing
any of them.
> -ssl session_lifetime 300
Don't think that would have a large effect, especially if you
terminate the SSL in haproxy.
Cheers,
-Nick
On Fri, Sep 2, 2022 at 6:02 AM Al Z. <az...@gmail.com> wrote:
>
> Thank you very much Nick for the reply.
> Most appreciated.
>
> 1) I checked _active_tasks and it return empty array []
>
>
> 2) About +sbwt, we are running on physical hardware ... no container.
>
> +sbwt none
> +S ${SCHEDULERS}
>
> Would the above config changes you suggested earlier have any adverse
> effect?
>
>
> 3) In my config file, there is
>
> # Set maximum SSL session lifetime to reap terminated replication readers
> -ssl session_lifetime 300
>
> Note that I am using HAProxy for SSL.
> Could this have any adverse effect?
>
> Thank you very much.
>
> Alfred.
>
>
> On Thu, Sep 1, 2022 at 6:02 AM Nick Vatamaniuc <va...@gmail.com> wrote:
>
> > Hi Alfred,
> >
> > It could be you're seeing the effect of busy waiting from Erlang VM
> > schedulers. Erlang VM would spawn a scheduler for each CPU core, and
> > then after any of those schedulers run out of work, they would still
> > spin in a busy loop for a bit waiting for more work. Sometimes that is
> > not desirable, especially in containerized environments so then it's
> > recommended to disable busy waiting by setting +sbwt to none in the
> > vm.args file. Give that a try and see if it helps. More info about it
> > can be found at https://adoptingerlang.org/docs/production/kubernetes/
> >
> > Another reason could be that there are some background index building
> > or compaction jobs running.. Check out the output of _active_tasks to
> > see if that's the case.
> >
> > Cheers,
> > -Nick
> >
> > On Sun, Aug 14, 2022 at 11:38 AM Al Z. <az...@gmail.com> wrote:
> > >
> > > Hello.
> > > I have the latest CouchDB 3.2.2 on Ubuntu 22.04
> > >
> > > Even when there are no active clients talking to couchdb, the CPU is
> > always
> > > above 15%.
> > > Not sure why.
> > > couchdb's beam.smp is currently the busiest process on the server.
> > >
> > > [image: image.png]
> > >
> > >
> > > # ps aux | grep couchdb
> > > couchdb 979 13.4 0.1 4647432 100284 ? Ssl 15:03 2:10
> > > /opt/couchdb/bin/../erts-12.2.1/bin/beam.smp -K true -A 16 -Bd -- -root
> > > /opt/couchdb/bin/.. -progname couchdb -- -home /opt/couchdb -- -boot
> > > /opt/couchdb/bin/../releases/3.2.2/couchdb -name couchdb@localhost
> > > -setcookie SOME_SECRET_COOKIE -kernel inet_dist_use_interface {127,0,0,1}
> > > -kernel error_logger silent -sasl sasl_error_logger false -noshell
> > -noinput
> > > -smp enable -ssl session_lifetime 300 -- -config
> > > /opt/couchdb/bin/../releases/3.2.2/sys.config --
> > > couchdb 1111 0.0 0.0 7136 220 ? S 15:03 0:00
> > > /opt/couchdb/bin/../erts-12.2.1/bin/epmd -daemon
> > > couchdb 1163 0.0 0.0 2780 972 ? Ss 15:03 0:00
> > > erl_child_setup 65536
> > > couchdb 2727 0.0 0.0 3736 1192 ? Ss 15:04 0:00
> > > inet_gethost 4
> > > couchdb 2728 0.0 0.0 3736 108 ? S 15:04 0:00
> > > inet_gethost 4
> > > root 2982 0.0 0.0 6612 2220 pts/1 S+ 15:30 0:00 grep
> > > --color=auto couchdb
> > >
> > >
> > > Any hint will be appreciated.
> > >
> > > Thank you very much.
> > >
> > > Alfred.
> >
Re: CPU constantly above 15% even when no client is talking to CouchDB
Posted by "Al Z." <az...@gmail.com>.
Thank you very much Nick for the reply.
Most appreciated.
1) I checked _active_tasks and it return empty array []
2) About +sbwt, we are running on physical hardware ... no container.
+sbwt none
+S ${SCHEDULERS}
Would the above config changes you suggested earlier have any adverse
effect?
3) In my config file, there is
# Set maximum SSL session lifetime to reap terminated replication readers
-ssl session_lifetime 300
Note that I am using HAProxy for SSL.
Could this have any adverse effect?
Thank you very much.
Alfred.
On Thu, Sep 1, 2022 at 6:02 AM Nick Vatamaniuc <va...@gmail.com> wrote:
> Hi Alfred,
>
> It could be you're seeing the effect of busy waiting from Erlang VM
> schedulers. Erlang VM would spawn a scheduler for each CPU core, and
> then after any of those schedulers run out of work, they would still
> spin in a busy loop for a bit waiting for more work. Sometimes that is
> not desirable, especially in containerized environments so then it's
> recommended to disable busy waiting by setting +sbwt to none in the
> vm.args file. Give that a try and see if it helps. More info about it
> can be found at https://adoptingerlang.org/docs/production/kubernetes/
>
> Another reason could be that there are some background index building
> or compaction jobs running.. Check out the output of _active_tasks to
> see if that's the case.
>
> Cheers,
> -Nick
>
> On Sun, Aug 14, 2022 at 11:38 AM Al Z. <az...@gmail.com> wrote:
> >
> > Hello.
> > I have the latest CouchDB 3.2.2 on Ubuntu 22.04
> >
> > Even when there are no active clients talking to couchdb, the CPU is
> always
> > above 15%.
> > Not sure why.
> > couchdb's beam.smp is currently the busiest process on the server.
> >
> > [image: image.png]
> >
> >
> > # ps aux | grep couchdb
> > couchdb 979 13.4 0.1 4647432 100284 ? Ssl 15:03 2:10
> > /opt/couchdb/bin/../erts-12.2.1/bin/beam.smp -K true -A 16 -Bd -- -root
> > /opt/couchdb/bin/.. -progname couchdb -- -home /opt/couchdb -- -boot
> > /opt/couchdb/bin/../releases/3.2.2/couchdb -name couchdb@localhost
> > -setcookie SOME_SECRET_COOKIE -kernel inet_dist_use_interface {127,0,0,1}
> > -kernel error_logger silent -sasl sasl_error_logger false -noshell
> -noinput
> > -smp enable -ssl session_lifetime 300 -- -config
> > /opt/couchdb/bin/../releases/3.2.2/sys.config --
> > couchdb 1111 0.0 0.0 7136 220 ? S 15:03 0:00
> > /opt/couchdb/bin/../erts-12.2.1/bin/epmd -daemon
> > couchdb 1163 0.0 0.0 2780 972 ? Ss 15:03 0:00
> > erl_child_setup 65536
> > couchdb 2727 0.0 0.0 3736 1192 ? Ss 15:04 0:00
> > inet_gethost 4
> > couchdb 2728 0.0 0.0 3736 108 ? S 15:04 0:00
> > inet_gethost 4
> > root 2982 0.0 0.0 6612 2220 pts/1 S+ 15:30 0:00 grep
> > --color=auto couchdb
> >
> >
> > Any hint will be appreciated.
> >
> > Thank you very much.
> >
> > Alfred.
>