You are viewing a plain text version of this content. The canonical link for it is here.

Posted to general@lucene.apache.org by Edward Quick <ed...@hotmail.com> on 2005/09/11 22:16:50 UTC

crawling protected pages

Hi,

I need to crawl a site that is protected. Is there currently any way to do 
this with nutch (i.e. providing a username/password) ?

Thanks,

Ed.

Re: crawling protected pages

Posted by Otis Gospodnetic <ot...@yahoo.com>.

I don't think there is.  I assume you are considering using Nutch.  If
so, use nutch-user mailing list instead.  You'll get more help there.

Otis

--- Edward Quick <ed...@hotmail.com> wrote:

> Hi,
> 
> I need to crawl a site that is protected. Is there currently any way
> to do 
> this with nutch (i.e. providing a username/password) ?
> 
> Thanks,
> 
> Ed.
> 
> 
>