You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Gianni Parini <gi...@gmail.com> on 2006/09/18 02:15:54 UTC

nutch can't crawl page redirected from port 80 to 8080

I everybody,

      i'm an italian student(informatic engineering) and i'm making  
thesis on search engine.
  At the end of that i'm trying to configure my university web server  
to work with nutch.
But i'm having some problem.

All the site is build in java jsp/jspx and the server is a apache  
tomcat reacheble on port 8080.

the site is http://www.alice.unibo.it and the first page redirect the  
browser
to http://www.alice.unibo.it:8080/index.jspx

The question's is that i need to index that web site, i configure  
nutch on the web server configuring:
-conf/nutch-site.xml
-url/nutch with http://www.alice.unibo.it
-conf/crawl-urlfilter.txt with alice.unibo.it for domain

And nutch cant't do that, i try 1000 way, putting the port in the  
filter domain,
putting the redirected url in the url/nutch file, modifying conf/ 
nutch-default.xml
plugin properties and other.

Nutch work perfectly with other domain without redirection,
but with redirection to 8080 can't fetch pages.
I Hope to find a solution in time to end discuss my thesis.

Thank you all.

Parini Gianni, Bologna, Italy



System env:
nutch 0.8
tomcat 5
java 1.5

System hardware:
MacOs 10.4.7 on Ibook G4