hardware config / problems

[Ba...@gmx.de]

hi everyone,

i'm new to nutch and i have some trouble to get a good working nutch-cluster setup.. (nutch 1.2)

my setup:
1 master (namenode, jobtracker, secondarynamenode)
2 nodes (datanode, tasktracker)

all pc's are virtual machines and have 500mb ram each

nutch-config:
- mapred.map.tasks 2
- mapred.reduce.tasks 2
- mapred.child.java.opts -Xmx256mb 
- fetcher.threads.fetch 20
- fetcher.server.delay 1.0
- fetcher.threads.per.host 3 
- replication 2

if i increase the map and reduce tasks the crawl doesn't work..because the tasktracker kills himself..(sometimes also the datanode)..so i changed it back to 2..

but my questions are more general..looks my setup/config ok? and is there any best practice hardware requirement for running a namenode or datanodes?

thanks for help and i appreciate all your answers..

bart


-- 
Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de