You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@nutch.apache.org by Lewis John Mcgibbney <le...@gmail.com> on 2015/10/01 20:19:27 UTC

Running JProfiler on Selenium Grid to Prevent System Overload

Hi Folks,
We've noticed that when running Selenium Grid with Nutch we can easily kill
an Amazon cluster. Amazon basically mention that load can get too high on
the frontend with "...too many java, firefox, and screen processes" which
stopped the EMR healthchecks from properly working. This caused the EMR
"logic" to terminate the cluster.
I wonder if anyone is or has undertaken JProfiling of Selenium + Nutch
particularly on the Selenium Grid configuration.
Thanks
Lewis

-- 
*Lewis*