You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@nutch.apache.org by Otis Gospodnetić <ot...@gmail.com> on 2016/01/31 23:35:41 UTC

DNS caching best practices

Hi,

The first item on http://wiki.apache.org/nutch/OptimizingCrawls is DNS
caching.  Is this still something people regularly do?  Even when running
in EC2, which I assume has nameservers that are relatively close to
instances doing crawling and nameserver lookups?

If so, are there any recommendations for the best DNS caching server/config
to use?

Thanks,
Otis
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/