URL verification

[LeenaB <le...@synechron.com>]

Hi,

I'm not sure if it is right forum for my question. But this is the question
for sitemap files required by Google search engine.

My application generates a file having thousands of url to access a various
section of my site. (sitemap urls required by search engine.)
So when I generate this file as it is having thousands of uRS it is
difficult to go and check each and every url if it is up and valid or no.

So I have written one small utility which parse this sitemap url file and
checks if urls are up or no and generates the report.
I use apache commons library for this:

{code}
                                     HttpClient client = new HttpClient();
					HttpMethod method = new GetMethod(urlToBeChecked);
					int statusCode = client.executeMethod(method);
					String responseCode=new Integer(statusCode).toString();
					if(!responseCode.startsWith("2")){
						*// add into the list of invalid url*
					}
					method.releaseConnection();
{code}

But this is very slow process; going and hitting each and every url. Is
there any faster method to do the same?

Regard,
Leena
-- 
View this message in context: http://www.nabble.com/URL-verification-tp24844563p24844563.html
Sent from the Commons - Issues mailing list archive at Nabble.com.

Re: URL verification

[sebb <se...@gmail.com>]

On 06/08/2009, LeenaB <le...@synechron.com> wrote:
>
>  Hi,
>
>  I'm not sure if it is right forum for my question.

No, it's not.

Please see:

http://hc.apache.org/mail.html

>  But this is the question
>  for sitemap files required by Google search engine.
>
>  My application generates a file having thousands of url to access a various
>  section of my site. (sitemap urls required by search engine.)
>  So when I generate this file as it is having thousands of uRS it is
>  difficult to go and check each and every url if it is up and valid or no.
>
>  So I have written one small utility which parse this sitemap url file and
>  checks if urls are up or no and generates the report.
>  I use apache commons library for this:
>
>  {code}
>                                      HttpClient client = new HttpClient();
>                                         HttpMethod method = new GetMethod(urlToBeChecked);
>                                         int statusCode = client.executeMethod(method);
>                                         String responseCode=new Integer(statusCode).toString();
>                                         if(!responseCode.startsWith("2")){
>                                                 *// add into the list of invalid url*
>                                         }
>                                         method.releaseConnection();
>  {code}
>
>  But this is very slow process; going and hitting each and every url. Is
>  there any faster method to do the same?
>
>  Regard,
>  Leena
>
> --
>  View this message in context: http://www.nabble.com/URL-verification-tp24844563p24844563.html
>  Sent from the Commons - Issues mailing list archive at Nabble.com.
>
>