You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Dan Volfman <da...@gmail.com> on 2012/01/24 18:22:19 UTC

Is it possible to go through a whole website but index only a specific file type?

I want to crawl a full website in order to index specific file types.
Assume I have a web site with many pages, some of them contain links to pdf
files.
I want to index only these files.
If I use suffix-filter to filter out   anything but .pdf, then nothing is
fetched (well I assume that its obvious - the pages that contain the pdf
links are not .pdf)

Is  it possible to tell nutch - go through the given url, take only links
to .pdf file and then index only the files, without indexing the pages
themselves?