You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@jmeter.apache.org by Richard Martin <bo...@googlemail.com> on 2007/06/20 23:37:25 UTC
Spidering a site, well testing
Hello,
I'm trying to use Jmeter to stress test a intermediary proxy to a site,
actually it will be used for many sites. To test it I need to move around
the site and I was hoping to use the HTML Link Parser to spider the site and
leave the clients/threads to work there way around. This allows me to easily
plug in other sites to test.
In searching I found a guide to doing this in the old version of Jmeter on
the old user guide. There is also a section about spidering under the help
documentation. It suggest putting in .* for path and domain. I tried this
with a Default HTTP containing the domain/host. An HTTP Request with .* and
a Link Parser but from my log it is just trying to request the page
"host/.*" which doesn't work. There was only so many ways I could arrange
these 3 things and now I'm out of ideas.
Am I missing something on this setup?
>From the help
"Spidering Example
Consider a simple example: let's say you wanted JMeter to "spider" through
your site, hitting link after link parsed from the HTML returned from your
server (this is not actually the most useful thing to do, but it serves as a
good example). You would create a Simple Controller , and add the "HTML Link
Parser" to it. Then, create an HTTP Request, and set the domain to ".*", and
the path likewise. This will cause your test sample to match with any link
found on the returned pages. If you wanted to restrict the spidering to a
particular domain, then change the domain value to the one you want. Then,
only links to that domain will be followed."
Thanks for any help
cheers
Bob
Re: Spidering a site, well testing (need clarification)
Posted by MGilly <gi...@gmail.com>.
Can someone explain sebb-2's answer in terms that a non-programmer who's
relatively new to JMeter can understand? Preferably with pictures? ;)
This is exactly what I need to do - go through all the links on my very
large ecommerce site. My developer wants to use JMeter rather than a simple
link checker (like Xenu) so that we can measure the performance of the
server as the program gets deeper into the links and more product pages are
cached, etc; as well as give us more control over how quickly the requests
are sent and in what order, etc.
Is sebb-2 saying that you have to provide all of the links in the first HTTP
Request, for example maybe using a list in a text file; or do you just need
to provide one link that is an example or something?
sebb-2 wrote:
>
> Thread Group
> + Http Request containing links
> + Http Request - host=.*, path=.*
> + + Link Parser
>
> The Link Parser needs to match the wild-card URL in the second request
> against the links in the first request.
>
> On 20/06/07, Richard Martin <bo...@googlemail.com> wrote:
>> Hello,
>>
>> I'm trying to use Jmeter to stress test a intermediary proxy to a site,
>> actually it will be used for many sites. To test it I need to move around
>> the site and I was hoping to use the HTML Link Parser to spider the site
>> and
>> leave the clients/threads to work there way around. This allows me to
>> easily
>> plug in other sites to test.
> [snip]
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>
>
--
View this message in context: http://www.nabble.com/Spidering-a-site%2C-well-testing-tf3955187.html#a11253996
Sent from the JMeter - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
Re: Spidering a site, well testing
Posted by sebb <se...@gmail.com>.
Thread Group
+ Http Request containing links
+ Http Request - host=.*, path=.*
+ + Link Parser
The Link Parser needs to match the wild-card URL in the second request
against the links in the first request.
On 20/06/07, Richard Martin <bo...@googlemail.com> wrote:
> Hello,
>
> I'm trying to use Jmeter to stress test a intermediary proxy to a site,
> actually it will be used for many sites. To test it I need to move around
> the site and I was hoping to use the HTML Link Parser to spider the site and
> leave the clients/threads to work there way around. This allows me to easily
> plug in other sites to test.
>
> In searching I found a guide to doing this in the old version of Jmeter on
> the old user guide. There is also a section about spidering under the help
> documentation. It suggest putting in .* for path and domain. I tried this
> with a Default HTTP containing the domain/host. An HTTP Request with .* and
> a Link Parser but from my log it is just trying to request the page
> "host/.*" which doesn't work. There was only so many ways I could arrange
> these 3 things and now I'm out of ideas.
>
> Am I missing something on this setup?
>
> From the help
>
> "Spidering Example
>
> Consider a simple example: let's say you wanted JMeter to "spider" through
> your site, hitting link after link parsed from the HTML returned from your
> server (this is not actually the most useful thing to do, but it serves as a
> good example). You would create a Simple Controller , and add the "HTML Link
> Parser" to it. Then, create an HTTP Request, and set the domain to ".*", and
> the path likewise. This will cause your test sample to match with any link
> found on the returned pages. If you wanted to restrict the spidering to a
> particular domain, then change the domain value to the one you want. Then,
> only links to that domain will be followed."
>
> Thanks for any help
>
> cheers
> Bob
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
RE: Spidering a site, well testing
Posted by "Smits, Dolf" <do...@siemens.com>.
Hi,
Bit late, but maybe stillhelpfull.
You can also try this by using the foreach controller.
You start on the homepage, us an extractor to extract all the http links
on that page, and then uses the forEach controller to walk them alll
over.
You can do the same on al these second level pages you get, and by
placing enough forEach loops underneath you can walk over the complete
website.
Good luck,
Dolf
Dolf Smits
Senior Consultant Identity Management Solutions
070 333 3654
070 333 2511
06 55844837
Dolf.Smits@siemens.com
Let op: Vrijdags heb ik ouderschapsverlof en wordt uw mail niet gelezen.
Deze e-mail is uitsluitend bedoeld voor kennisneming door de
geadresseerde(n), en mag niet aan anderen worden doorgestuurd of op
andere wijze ter kennis worden gebracht. Indien u niet de geadresseerde
bent, verzoek ik u om de afzender te waarschuwen en de e-mail direct te
verwijderen/vernietigen. De afzender wijst elke aansprakelijkheid voor
(de inhoud van) deze e-mail af.
-----Original Message-----
From: Richard Martin [mailto:bobthekingofegypt@googlemail.com]
Sent: donderdag 21 juni 2007 13:12
To: JMeter Users List
Subject: Re: Spidering a site, well testing
Thanks guys, I found your version Marc. But I still have a problem,
with
the while loop it now works its way around in a random order (which I
actually like) but it spiders through, from say the first page, chooses
a
random link then follows it and does the same on the next page and the
next
page until it gets to a page with no links then it requests http://.*/.*
and
stops. This is bad because it generates an error that isn't really an
error. I can live with it stopping when reaching a page with no links,
it
would be better if it went back and tried a different route, but I can't
have it throwing errors.
Any ideas how to stop it making that final request for http://.*/.*
cheers
Bob
On 6/21/07, Marc Anthony Winoto <ma...@netregistry.com.au> wrote:
>
> I also tried going this as one of my first Jmeter projects. I sent my
> version to the mailing list because I could not get the doc example to
> work either.
>
> If you search the mailing list archive for my email, I am sure you
will
> find it.
>
>
> Richard Martin wrote:
> > Hello,
> >
> > I'm trying to use Jmeter to stress test a intermediary proxy to a
site,
> > actually it will be used for many sites. To test it I need to move
> around
> > the site and I was hoping to use the HTML Link Parser to spider the
site
> > and
> > leave the clients/threads to work there way around. This allows me
to
> > easily
> > plug in other sites to test.
> >
> > In searching I found a guide to doing this in the old version of
Jmeter
> on
> > the old user guide. There is also a section about spidering under
the
> help
> > documentation. It suggest putting in .* for path and domain. I tried
> this
> > with a Default HTTP containing the domain/host. An HTTP Request with
.*
> and
> > a Link Parser but from my log it is just trying to request the page
> > "host/.*" which doesn't work. There was only so many ways I could
> arrange
> > these 3 things and now I'm out of ideas.
> >
> > Am I missing something on this setup?
> >
> >> From the help
> >
> > "Spidering Example
> >
> > Consider a simple example: let's say you wanted JMeter to "spider"
> through
> > your site, hitting link after link parsed from the HTML returned
from
> your
> > server (this is not actually the most useful thing to do, but it
serves
> > as a
> > good example). You would create a Simple Controller , and add the
"HTML
> > Link
> > Parser" to it. Then, create an HTTP Request, and set the domain to
".*",
> > and
> > the path likewise. This will cause your test sample to match with
any
> link
> > found on the returned pages. If you wanted to restrict the spidering
to
> a
> > particular domain, then change the domain value to the one you want.
> Then,
> > only links to that domain will be followed."
> >
> > Thanks for any help
> >
> > cheers
> > Bob
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
Re: Spidering a site, well testing
Posted by Richard Martin <bo...@googlemail.com>.
Thanks guys, I found your version Marc. But I still have a problem, with
the while loop it now works its way around in a random order (which I
actually like) but it spiders through, from say the first page, chooses a
random link then follows it and does the same on the next page and the next
page until it gets to a page with no links then it requests http://.*/.* and
stops. This is bad because it generates an error that isn't really an
error. I can live with it stopping when reaching a page with no links, it
would be better if it went back and tried a different route, but I can't
have it throwing errors.
Any ideas how to stop it making that final request for http://.*/.*
cheers
Bob
On 6/21/07, Marc Anthony Winoto <ma...@netregistry.com.au> wrote:
>
> I also tried going this as one of my first Jmeter projects. I sent my
> version to the mailing list because I could not get the doc example to
> work either.
>
> If you search the mailing list archive for my email, I am sure you will
> find it.
>
>
> Richard Martin wrote:
> > Hello,
> >
> > I'm trying to use Jmeter to stress test a intermediary proxy to a site,
> > actually it will be used for many sites. To test it I need to move
> around
> > the site and I was hoping to use the HTML Link Parser to spider the site
> > and
> > leave the clients/threads to work there way around. This allows me to
> > easily
> > plug in other sites to test.
> >
> > In searching I found a guide to doing this in the old version of Jmeter
> on
> > the old user guide. There is also a section about spidering under the
> help
> > documentation. It suggest putting in .* for path and domain. I tried
> this
> > with a Default HTTP containing the domain/host. An HTTP Request with .*
> and
> > a Link Parser but from my log it is just trying to request the page
> > "host/.*" which doesn't work. There was only so many ways I could
> arrange
> > these 3 things and now I'm out of ideas.
> >
> > Am I missing something on this setup?
> >
> >> From the help
> >
> > "Spidering Example
> >
> > Consider a simple example: let's say you wanted JMeter to "spider"
> through
> > your site, hitting link after link parsed from the HTML returned from
> your
> > server (this is not actually the most useful thing to do, but it serves
> > as a
> > good example). You would create a Simple Controller , and add the "HTML
> > Link
> > Parser" to it. Then, create an HTTP Request, and set the domain to ".*",
> > and
> > the path likewise. This will cause your test sample to match with any
> link
> > found on the returned pages. If you wanted to restrict the spidering to
> a
> > particular domain, then change the domain value to the one you want.
> Then,
> > only links to that domain will be followed."
> >
> > Thanks for any help
> >
> > cheers
> > Bob
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>
Re: Spidering a site, well testing
Posted by Marc Anthony Winoto <ma...@netregistry.com.au>.
I also tried going this as one of my first Jmeter projects. I sent my
version to the mailing list because I could not get the doc example to
work either.
If you search the mailing list archive for my email, I am sure you will
find it.
Richard Martin wrote:
> Hello,
>
> I'm trying to use Jmeter to stress test a intermediary proxy to a site,
> actually it will be used for many sites. To test it I need to move around
> the site and I was hoping to use the HTML Link Parser to spider the site
> and
> leave the clients/threads to work there way around. This allows me to
> easily
> plug in other sites to test.
>
> In searching I found a guide to doing this in the old version of Jmeter on
> the old user guide. There is also a section about spidering under the help
> documentation. It suggest putting in .* for path and domain. I tried this
> with a Default HTTP containing the domain/host. An HTTP Request with .* and
> a Link Parser but from my log it is just trying to request the page
> "host/.*" which doesn't work. There was only so many ways I could arrange
> these 3 things and now I'm out of ideas.
>
> Am I missing something on this setup?
>
>> From the help
>
> "Spidering Example
>
> Consider a simple example: let's say you wanted JMeter to "spider" through
> your site, hitting link after link parsed from the HTML returned from your
> server (this is not actually the most useful thing to do, but it serves
> as a
> good example). You would create a Simple Controller , and add the "HTML
> Link
> Parser" to it. Then, create an HTTP Request, and set the domain to ".*",
> and
> the path likewise. This will cause your test sample to match with any link
> found on the returned pages. If you wanted to restrict the spidering to a
> particular domain, then change the domain value to the one you want. Then,
> only links to that domain will be followed."
>
> Thanks for any help
>
> cheers
> Bob
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org