[GitHub] [camel-kafka-connector] Thylossus commented on issue #414: High CPU Usage with RabbitMQ Connector (polling interface implemented incorrectly?)

[GitBox <gi...@apache.org>]

Thylossus commented on issue #414:
URL: https://github.com/apache/camel-kafka-connector/issues/414#issuecomment-686316516


   Hi,
   
   thank you very much for looking into the issue. I've prepared a docker-compose file to reproduce my issue: https://github.com/Thylossus/docker-kafka-camel-rabbitmq-connector-cpu-usage-scenario. I did not have the time yesterday to do so.
   
   When I start all container and setup the RabbitMQ connector, the CPU usage almost immediately increases significantly. I use top in Irix mode to check the resources:
   
   ```
   Mem: 4911524K used, 8137016K free, 1252K shrd, 43904K buff, 1593012K cached
   CPU:  84% usr   0% sys   0% nic  15% idle   0% io   0% irq   0% sirq
   Load average: 10.14 6.92 3.24 12/1265 473
     PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
       6     1 root     S    2688m  21%   2  79% /usr/lib/jvm/java-1.8-openjdk/jre/bin/java -Xms256M -Xmx1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesConcurrent -Djava.awt.headless=true -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dkafka.logs.dir=/kafka/connect/app/bin/../logs -Dlog4j.configuration=file:/ka
       1     0 root     S     2180   0%   5   0% bash /kafka/connect/entrypoint.sh
     462     0 root     S     1532   0%   2   0% top
     467     0 root     R     1528   0%   6   0% top -b -n 1
   ```
   
   However, on our test systems we also have New Relic Monitoring which reports high CPU usage as well.
   
   With the docker-compose setup on my local machine, you can also see the heavy load on the host system:
   
   ![grafik](https://user-images.githubusercontent.com/6013423/92085615-5e07cb80-edc9-11ea-93e1-687e63bcefa2.png)
   
   Maybe the cause lies within my connector configuration (I've also tried adding `camel.source.maxPollDuration=10000` as in your configuration, but it did not change the behavior) or running Kafka Connect in distributed mode. I've spent quite some time investigating this issue and cannot explain what's the cause of the high CPU usage except for my assumption in the initial issue description.
   
   Best regards,
   
   Tobias
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org