You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by jeevi tesh <je...@gmail.com> on 2015/02/14 18:04:59 UTC
HTTPClient is not loading a page
Hi,
Below URL's web page content I'm not able to get using httpclient. Please
help me in this regard
http://parts.arrow.com/item/search/?utm_campaign=arrowMain&utm_medium=link&utm_source=arrow&utm_content=www.arrow.com#st=ABLS-7.3728MHZ-B2-T;reNwlT
Here is my piece of code
HttpGet httget = new HttpGet("
http://parts.arrow.com/item/search/?utm_campaign=arrowMain&utm_medium=link&utm_source=arrow&utm_content=www.arrow.com#st=ABLS-7.3728MHZ-B2-T;reNwlT
");
HttpClient httpclient = new DefaultHttpClient();
HttpResponse response = httpclient.execute(httget);
HttpEntity entity = response.getEntity();
String thirdPartyPageContent=EntityUtils.toString(entity);
System.out.println(thirdPartyPageContent);
I'm not getting actual page content.
If any insight will be greatly appreciated.
Thanks
jeevitesh
Re: HTTPClient is not loading a page
Posted by Stefan Magnus Landrø <st...@gmail.com>.
Httpclient doesn't know how to handle JavaScript or Ajax. You'll have to parse the response for urls etc and perform these Ajax calls "yourself"
Sendt fra min iPhone
> Den 16. feb. 2015 kl. 19.47 skrev jeevi tesh <je...@gmail.com>:
>
> I added header and still not working..
> below is my code. When I use firefoxdriver it work.. Looks like making ajax
> call but even after looking for several hours we not able to confirm that
> because of ajax call we were not able to load. Any idea how to figure out
> ajax calls are made or not.
> Looking forward for replay
> thanks
> jeevitesh
>
>
>
> HttpGet httget = new HttpGet("
> http://parts.arrow.com/item/search/?utm_campaign=arrowMain&utm_medium=link&utm_source=arrow&utm_content=www.arrow.com#st=ABLS-7.3728MHZ-B2-T;reNwlT
> ");
> HttpClient httpclient = HttpClientBuilder.create().setRedirectStrategy(new
> LaxRedirectStrategy()).build();
> httget.setHeader("User-Agent", "MySuperUserAgent");
> HttpResponse response = httpclient.execute(httget);
> HttpEntity entity = response.getEntity();
> String thirdPartyPageContent=EntityUtils.toString(entity);
>
> On Sat, Feb 14, 2015 at 11:15 PM, Stefan Magnus Landrø <
> stefan.landro@gmail.com> wrote:
>
>> Maybe use a different user-agent header?
>>
>> Sendt fra min iPhone
>>
>>> Den 14. feb. 2015 kl. 18.04 skrev jeevi tesh <je...@gmail.com>:
>>>
>>> Hi,
>>> Below URL's web page content I'm not able to get using httpclient. Please
>>> help me in this regard
>> http://parts.arrow.com/item/search/?utm_campaign=arrowMain&utm_medium=link&utm_source=arrow&utm_content=www.arrow.com#st=ABLS-7.3728MHZ-B2-T;reNwlT
>>>
>>> Here is my piece of code
>>>
>>> HttpGet httget = new HttpGet("
>> http://parts.arrow.com/item/search/?utm_campaign=arrowMain&utm_medium=link&utm_source=arrow&utm_content=www.arrow.com#st=ABLS-7.3728MHZ-B2-T;reNwlT
>>> ");
>>>
>>> HttpClient httpclient = new DefaultHttpClient();
>>> HttpResponse response = httpclient.execute(httget);
>>> HttpEntity entity = response.getEntity();
>>> String thirdPartyPageContent=EntityUtils.toString(entity);
>>> System.out.println(thirdPartyPageContent);
>>>
>>> I'm not getting actual page content.
>>>
>>> If any insight will be greatly appreciated.
>>>
>>> Thanks
>>> jeevitesh
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: HTTPClient is not loading a page
Posted by jeevi tesh <je...@gmail.com>.
I added header and still not working..
below is my code. When I use firefoxdriver it work.. Looks like making ajax
call but even after looking for several hours we not able to confirm that
because of ajax call we were not able to load. Any idea how to figure out
ajax calls are made or not.
Looking forward for replay
thanks
jeevitesh
HttpGet httget = new HttpGet("
http://parts.arrow.com/item/search/?utm_campaign=arrowMain&utm_medium=link&utm_source=arrow&utm_content=www.arrow.com#st=ABLS-7.3728MHZ-B2-T;reNwlT
");
HttpClient httpclient = HttpClientBuilder.create().setRedirectStrategy(new
LaxRedirectStrategy()).build();
httget.setHeader("User-Agent", "MySuperUserAgent");
HttpResponse response = httpclient.execute(httget);
HttpEntity entity = response.getEntity();
String thirdPartyPageContent=EntityUtils.toString(entity);
On Sat, Feb 14, 2015 at 11:15 PM, Stefan Magnus Landrø <
stefan.landro@gmail.com> wrote:
> Maybe use a different user-agent header?
>
> Sendt fra min iPhone
>
> > Den 14. feb. 2015 kl. 18.04 skrev jeevi tesh <je...@gmail.com>:
> >
> > Hi,
> > Below URL's web page content I'm not able to get using httpclient. Please
> > help me in this regard
> >
> http://parts.arrow.com/item/search/?utm_campaign=arrowMain&utm_medium=link&utm_source=arrow&utm_content=www.arrow.com#st=ABLS-7.3728MHZ-B2-T;reNwlT
> >
> > Here is my piece of code
> >
> > HttpGet httget = new HttpGet("
> >
> http://parts.arrow.com/item/search/?utm_campaign=arrowMain&utm_medium=link&utm_source=arrow&utm_content=www.arrow.com#st=ABLS-7.3728MHZ-B2-T;reNwlT
> > ");
> >
> > HttpClient httpclient = new DefaultHttpClient();
> > HttpResponse response = httpclient.execute(httget);
> > HttpEntity entity = response.getEntity();
> > String thirdPartyPageContent=EntityUtils.toString(entity);
> > System.out.println(thirdPartyPageContent);
> >
> > I'm not getting actual page content.
> >
> > If any insight will be greatly appreciated.
> >
> > Thanks
> > jeevitesh
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
Re: HTTPClient is not loading a page
Posted by Stefan Magnus Landrø <st...@gmail.com>.
Maybe use a different user-agent header?
Sendt fra min iPhone
> Den 14. feb. 2015 kl. 18.04 skrev jeevi tesh <je...@gmail.com>:
>
> Hi,
> Below URL's web page content I'm not able to get using httpclient. Please
> help me in this regard
> http://parts.arrow.com/item/search/?utm_campaign=arrowMain&utm_medium=link&utm_source=arrow&utm_content=www.arrow.com#st=ABLS-7.3728MHZ-B2-T;reNwlT
>
> Here is my piece of code
>
> HttpGet httget = new HttpGet("
> http://parts.arrow.com/item/search/?utm_campaign=arrowMain&utm_medium=link&utm_source=arrow&utm_content=www.arrow.com#st=ABLS-7.3728MHZ-B2-T;reNwlT
> ");
>
> HttpClient httpclient = new DefaultHttpClient();
> HttpResponse response = httpclient.execute(httget);
> HttpEntity entity = response.getEntity();
> String thirdPartyPageContent=EntityUtils.toString(entity);
> System.out.println(thirdPartyPageContent);
>
> I'm not getting actual page content.
>
> If any insight will be greatly appreciated.
>
> Thanks
> jeevitesh
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org