You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Chetan Sahasrabudhe <Ch...@KPITCummins.com> on 2005/06/06 14:23:22 UTC
RE: proxy
Hi,
I have been trying to use the patch httpclient so as to enable fetcher operations through proxy.
When I try to compile the files, class http is looking for following imports
import org.apache.nutch.protocol.ProtocolOutput;
import org.apache.nutch.protocol.ProtocolStatus;
I have downloaded current build from nutch nightly downloads.
havnt got any lucky in locating these two files.
Andrzej, Piotr, Andy, Ken can anyone give me some pointers.
Regards
Chetan
_______________________________
Tel +91-20-5652 5000 ext 2513
KPITCummins Infosystems Limited
Hinjwadi
Pune INDIA
_______________________________
-----Original Message-----
From: Piotr Kosiorowski [mailto:pkosiorowski@gmail.com]
Sent: Wednesday, May 11, 2005 6:08 PM
To: nutch-user@incubator.apache.org
Subject: Re: proxy
Hello,
As far as I remember in current implementation it is not possible to
fetch through a proxy that requires authorization. But Andrzej Bialecki
is working on httpclient based implementation right now and as
httpclient allows one to use proxy with authorization it should be no
problem to do so in nutch.
You can check current status of httpclient based patch by looking for
thread: "Update: HTTPClient for protocol-http and protocol-https".
You can even try the patch yourself.
Regards
Piotr
k-team wrote:
> Hi all,
> I'm testing nutch on my PC, and need to get through a proxy to crawls pages.
> I've tried to set the "http.proxy.host" property like this:
> user:password@proxyIP
> but I get this error message:
> fetch of http://www.host.com/ failed with:
> net.nutch.protocol.http.HttpException: java.net.UnknownHostException:
> user:password@proxyIP
>
> How can I set my proxy with user/pwd? Is it possible?
>
> thanks,
> Kteam
Re: proxy
Posted by Doug Cutting <cu...@nutch.org>.
Piotr Kosiorowski wrote:
> Andrzej patch was integrated with latest SVN version, but I am not sure
> why nightly downloads on nutch page are quite old - so they do not
> contain it.
I just fixed the nightly build process. It runs on my home workstation.
Please send a message to the list if you notice again that it's
broken. Someday we should move it to a server at Apache.
Doug
Re: proxy
Posted by Piotr Kosiorowski <pk...@gmail.com>.
Hello Chetan,
Andrzej patch was integrated with latest SVN version, but I am not sure
why nightly downloads on nutch page are quite old - so they do not
contain it. So you should download the source from SVN and compile it:
http://lucene.apache.org/nutch/version_control.html
I am not sure if proxy authorization is already working or it was not
included in the patch - I had no time to check it.
Regards,
Piotr
Chetan Sahasrabudhe wrote:
> Hi,
>
> I have been trying to use the patch httpclient so as to enable fetcher operations through proxy.
>
> When I try to compile the files, class http is looking for following imports
>
> import org.apache.nutch.protocol.ProtocolOutput;
> import org.apache.nutch.protocol.ProtocolStatus;
>
> I have downloaded current build from nutch nightly downloads.
> havnt got any lucky in locating these two files.
>
> Andrzej, Piotr, Andy, Ken can anyone give me some pointers.
>
> Regards
> Chetan
> _______________________________
>
> Tel +91-20-5652 5000 ext 2513
>
> KPITCummins Infosystems Limited
> Hinjwadi
> Pune INDIA
> _______________________________
>
>
>
> -----Original Message-----
> From: Piotr Kosiorowski [mailto:pkosiorowski@gmail.com]
> Sent: Wednesday, May 11, 2005 6:08 PM
> To: nutch-user@incubator.apache.org
> Subject: Re: proxy
>
>
> Hello,
> As far as I remember in current implementation it is not possible to
> fetch through a proxy that requires authorization. But Andrzej Bialecki
> is working on httpclient based implementation right now and as
> httpclient allows one to use proxy with authorization it should be no
> problem to do so in nutch.
> You can check current status of httpclient based patch by looking for
> thread: "Update: HTTPClient for protocol-http and protocol-https".
> You can even try the patch yourself.
> Regards
> Piotr
>
>
> k-team wrote:
>
>>Hi all,
>>I'm testing nutch on my PC, and need to get through a proxy to crawls pages.
>>I've tried to set the "http.proxy.host" property like this:
>>user:password@proxyIP
>>but I get this error message:
>>fetch of http://www.host.com/ failed with:
>>net.nutch.protocol.http.HttpException: java.net.UnknownHostException:
>>user:password@proxyIP
>>
>>How can I set my proxy with user/pwd? Is it possible?
>>
>>thanks,
>>Kteam
>
>
Re: proxy
Posted by Andrzej Bialecki <ab...@getopt.org>.
Chetan Sahasrabudhe wrote:
> Hi,
>
> I have been trying to use the patch httpclient so as to enable fetcher operations through proxy.
>
> When I try to compile the files, class http is looking for following imports
>
> import org.apache.nutch.protocol.ProtocolOutput;
> import org.apache.nutch.protocol.ProtocolStatus;
>
> I have downloaded current build from nutch nightly downloads.
> havnt got any lucky in locating these two files.
>
> Andrzej, Piotr, Andy, Ken can anyone give me some pointers.
These two have been committed in SVN r179436. Please make sure you have
the same (or later) revision, and then run "ant clean compile".
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com