You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@jmeter.apache.org by Sonam Chauhan <so...@ce.com.au> on 2010/04/28 05:57:11 UTC
Questions on Jmeter reporting?
Hello all:
Just wanted to ask your thoughts on what to use for analysis and reporting of JMeter test runs.
I read the log analysis options on the Wiki: http://wiki.apache.org/jakarta-jmeter/LogAnalysis. I also added an entry about the Hudson continuous integration server's JMeter plug-in.
However, most solutions there are for JMeter reporting "in-the-large" - i.e. they paint the large picture on how the system copes with load.
Is there a JMeter reporting solution that lets you drill down to specific errors generated by specific samplers? For instance, down to this level of detail:
JMeter Testcase ABC.jmx >
Test Run # 12 >
Thread Group 1, Thread #12 >
Iteration 2 >
Sampler XYZ - Timestamp 2010-04-20 09:01:05 >
[ Assertion Failure Messages ]
[ HTTP Response Body ]
JMeter logs assertion failures by default, and can be configured to capture the full sampler response on error, so this should be possible. I just wanted to ask if any out-of-the-box solution implements this degree of detail.
Regards
Sonam
The information contained in this email and any attached files are strictly
private and confidential. This email should be read by the intended addressee
only. If the recipient of this message is not the intended addressee, please
call Corporate Express Australia Limited on +61 2 9335 0555 or Corporate Express
New Zealand Limited on +64 9 271 7600 and promptly delete this email and any
attachments. The intended recipient of this email may only use, reproduce,
disclose or distribute the information contained in this email and any attached
files with Corporate Express' permission. If you are not the intended addressee,
you are strictly prohibited from using, reproducing, disclosing or distributing
the information contained in this email and any attached files. Corporate
Express advises that this email and any attached files should be scanned to
detect viruses. Corporate Express accepts no liability for loss or damage
(whether caused by negligence or not) resulting from the use of any attached
files.
Re: Questions on Jmeter reporting?
Posted by sebb <se...@gmail.com>.
On 03/05/2010, Sonam Chauhan <so...@ce.com.au> wrote:
> Sorry, I sent my response before it was ready. Yes, that's a good idea.
>
>
>
> Another idea is setting the JMeter config to save the sampler response when an error occurs, and parsing out the response.
>
>
>
> I do this by using these-D parameters when running JMeter from the command line:
>
> ----------
>
> -Dfile_format.testlog=2.0
>
> -Djmeter.save.saveservice.response_data.on_error=true
>
> -Djmeter.save.saveservice.requestHeaders=true
>
> ----------
>
>
>
> The resultant XML log does save the response data, but the XML is no longer well-formed.
I don't immediately see the problem with the XML - if the data is not
being encoded correctly before being saved, that ought to be fixed.
Perhaps you could raise a Bugzilla for the problem? Thanks.
> ----------
>
> POST data:
>
> code=123123123& quantity=2&addtoorder=yes
>
> Cookie Data:
>
> sess=608205ej24902733270b2b23;
>
> ...
>
> <assertionResult failureMessage="Test failed: text expected to contain /1 product added/" error="false" failure="true"/>
>
> <binary>
>
> ...[HTML-encoded response webpage]...
>
> </binary>
>
> ----------
>
>
>
> Regards
>
>
> Sonam
>
>
> -----Original Message-----
> From: Sonam Chauhan [mailto:sonam.chauhan@ce.com.au]
>
> Sent: Monday, 3 May 2010 4:16 PM
> To: JMeter Users List
>
> Subject: RE: Questions on Jmeter reporting?
>
>
>
> That's a good idea Steve!
>
>
>
> Anopther
>
>
>
>
>
> -----Original Message-----
>
> From: Steve Eckhart [mailto:Steve.Eckhart@wausaufs.com]
>
> Sent: Saturday, 1 May 2010 5:35 AM
>
> To: JMeter Users List
>
> Subject: RE: Questions on Jmeter reporting?
>
>
>
> Sonam,
>
>
>
> What I have done is to include identifying information in the Name of the Response Assertion. For example: Batch Summary Search Error ${RowNumber}: ${LogonName} - ${LockboxID} - ${StartDate} - ${EndDate}
>
>
>
> The variables are entries in the CSV data files used to create the test. Then, I can go back and try to recreate the error in a browser. This lets me know whether there is a problem in the application (the error appears in a browser, too), in the script not generating the request correctly, or if the error was caused by the load.
>
>
>
>
>
> Steve Eckhart
>
> Business Analyst/Project Manager
>
>
>
> WAUSAU Financial Systems
>
> 14010 FNB PKWY STE500
>
> Omaha, NE 68154-5206
>
>
>
> Enterprise Payment Solutions.
>
> CONFIDENTIALITY NOTICE: This electronic transmission (including files attached hereto) is intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any disclosure, dissemination, copying, distribution or taking of any action in reliance on the contents of this confidential information is strictly prohibited. If you have received this communication in error, please destroy it and immediately notify us by return email. Thank you.
>
>
>
>
>
> -----Original Message-----
>
> From: Sonam Chauhan [mailto:sonam.chauhan@ce.com.au]
>
> Sent: Tuesday, April 27, 2010 10:57 PM
>
> To: JMeter Users List
>
> Subject: Questions on Jmeter reporting?
>
>
>
> Hello all:
>
>
>
> Just wanted to ask your thoughts on what to use for analysis and reporting of JMeter test runs.
>
>
>
> I read the log analysis options on the Wiki: http://wiki.apache.org/jakarta-jmeter/LogAnalysis. I also added an entry about the Hudson continuous integration server's JMeter plug-in.
>
>
>
> However, most solutions there are for JMeter reporting "in-the-large" - i.e. they paint the large picture on how the system copes with load.
>
>
>
> Is there a JMeter reporting solution that lets you drill down to specific errors generated by specific samplers? For instance, down to this level of detail:
>
> JMeter Testcase ABC.jmx >
>
> Test Run # 12 >
>
> Thread Group 1, Thread #12 >
>
> Iteration 2 >
>
> Sampler XYZ - Timestamp 2010-04-20 09:01:05 >
>
> [ Assertion Failure Messages ]
>
> [ HTTP Response Body ]
>
>
>
> JMeter logs assertion failures by default, and can be configured to capture the full sampler response on error, so this should be possible. I just wanted to ask if any out-of-the-box solution implements this degree of detail.
>
>
>
> Regards
>
> Sonam
>
>
>
>
>
>
>
>
>
> The information contained in this email and any attached files are strictly
>
> private and confidential. This email should be read by the intended addressee
>
> only. If the recipient of this message is not the intended addressee, please
>
> call Corporate Express Australia Limited on +61 2 9335 0555 or Corporate Express
>
> New Zealand Limited on +64 9 271 7600 and promptly delete this email and any
>
> attachments. The intended recipient of this email may only use, reproduce,
>
> disclose or distribute the information contained in this email and any attached
>
> files with Corporate Express' permission. If you are not the intended addressee,
>
> you are strictly prohibited from using, reproducing, disclosing or distributing
>
> the information contained in this email and any attached files. Corporate
>
> Express advises that this email and any attached files should be scanned to
>
> detect viruses. Corporate Express accepts no liability for loss or damage
>
> (whether caused by negligence or not) resulting from the use of any attached
>
> files.
>
>
>
> ---------------------------------------------------------------------
>
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
>
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
>
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
>
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
RE: Questions on Jmeter reporting?
Posted by Sonam Chauhan <so...@ce.com.au>.
Sorry, I sent my response before it was ready. Yes, that's a good idea.
Another idea is setting the JMeter config to save the sampler response when an error occurs, and parsing out the response.
I do this by using these-D parameters when running JMeter from the command line:
----------
-Dfile_format.testlog=2.0
-Djmeter.save.saveservice.response_data.on_error=true
-Djmeter.save.saveservice.requestHeaders=true
----------
The resultant XML log does save the response data, but the XML is no longer well-formed.
----------
POST data:
code=123123123& quantity=2&addtoorder=yes
Cookie Data:
sess=608205ej24902733270b2b23;
...
<assertionResult failureMessage="Test failed: text expected to contain /1 product added/" error="false" failure="true"/>
<binary>
...[HTML-encoded response webpage]...
</binary>
----------
Regards
Sonam
-----Original Message-----
From: Sonam Chauhan [mailto:sonam.chauhan@ce.com.au]
Sent: Monday, 3 May 2010 4:16 PM
To: JMeter Users List
Subject: RE: Questions on Jmeter reporting?
That's a good idea Steve!
Anopther
-----Original Message-----
From: Steve Eckhart [mailto:Steve.Eckhart@wausaufs.com]
Sent: Saturday, 1 May 2010 5:35 AM
To: JMeter Users List
Subject: RE: Questions on Jmeter reporting?
Sonam,
What I have done is to include identifying information in the Name of the Response Assertion. For example: Batch Summary Search Error ${RowNumber}: ${LogonName} - ${LockboxID} - ${StartDate} - ${EndDate}
The variables are entries in the CSV data files used to create the test. Then, I can go back and try to recreate the error in a browser. This lets me know whether there is a problem in the application (the error appears in a browser, too), in the script not generating the request correctly, or if the error was caused by the load.
Steve Eckhart
Business Analyst/Project Manager
WAUSAU Financial Systems
14010 FNB PKWY STE500
Omaha, NE 68154-5206
Enterprise Payment Solutions.
CONFIDENTIALITY NOTICE: This electronic transmission (including files attached hereto) is intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any disclosure, dissemination, copying, distribution or taking of any action in reliance on the contents of this confidential information is strictly prohibited. If you have received this communication in error, please destroy it and immediately notify us by return email. Thank you.
-----Original Message-----
From: Sonam Chauhan [mailto:sonam.chauhan@ce.com.au]
Sent: Tuesday, April 27, 2010 10:57 PM
To: JMeter Users List
Subject: Questions on Jmeter reporting?
Hello all:
Just wanted to ask your thoughts on what to use for analysis and reporting of JMeter test runs.
I read the log analysis options on the Wiki: http://wiki.apache.org/jakarta-jmeter/LogAnalysis. I also added an entry about the Hudson continuous integration server's JMeter plug-in.
However, most solutions there are for JMeter reporting "in-the-large" - i.e. they paint the large picture on how the system copes with load.
Is there a JMeter reporting solution that lets you drill down to specific errors generated by specific samplers? For instance, down to this level of detail:
JMeter Testcase ABC.jmx >
Test Run # 12 >
Thread Group 1, Thread #12 >
Iteration 2 >
Sampler XYZ - Timestamp 2010-04-20 09:01:05 >
[ Assertion Failure Messages ]
[ HTTP Response Body ]
JMeter logs assertion failures by default, and can be configured to capture the full sampler response on error, so this should be possible. I just wanted to ask if any out-of-the-box solution implements this degree of detail.
Regards
Sonam
The information contained in this email and any attached files are strictly
private and confidential. This email should be read by the intended addressee
only. If the recipient of this message is not the intended addressee, please
call Corporate Express Australia Limited on +61 2 9335 0555 or Corporate Express
New Zealand Limited on +64 9 271 7600 and promptly delete this email and any
attachments. The intended recipient of this email may only use, reproduce,
disclose or distribute the information contained in this email and any attached
files with Corporate Express' permission. If you are not the intended addressee,
you are strictly prohibited from using, reproducing, disclosing or distributing
the information contained in this email and any attached files. Corporate
Express advises that this email and any attached files should be scanned to
detect viruses. Corporate Express accepts no liability for loss or damage
(whether caused by negligence or not) resulting from the use of any attached
files.
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
RE: Questions on Jmeter reporting?
Posted by Sonam Chauhan <so...@ce.com.au>.
That's a good idea Steve!
Anopther
-----Original Message-----
From: Steve Eckhart [mailto:Steve.Eckhart@wausaufs.com]
Sent: Saturday, 1 May 2010 5:35 AM
To: JMeter Users List
Subject: RE: Questions on Jmeter reporting?
Sonam,
What I have done is to include identifying information in the Name of the Response Assertion. For example: Batch Summary Search Error ${RowNumber}: ${LogonName} - ${LockboxID} - ${StartDate} - ${EndDate}
The variables are entries in the CSV data files used to create the test. Then, I can go back and try to recreate the error in a browser. This lets me know whether there is a problem in the application (the error appears in a browser, too), in the script not generating the request correctly, or if the error was caused by the load.
Steve Eckhart
Business Analyst/Project Manager
WAUSAU Financial Systems
14010 FNB PKWY STE500
Omaha, NE 68154-5206
Enterprise Payment Solutions.
CONFIDENTIALITY NOTICE: This electronic transmission (including files attached hereto) is intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any disclosure, dissemination, copying, distribution or taking of any action in reliance on the contents of this confidential information is strictly prohibited. If you have received this communication in error, please destroy it and immediately notify us by return email. Thank you.
-----Original Message-----
From: Sonam Chauhan [mailto:sonam.chauhan@ce.com.au]
Sent: Tuesday, April 27, 2010 10:57 PM
To: JMeter Users List
Subject: Questions on Jmeter reporting?
Hello all:
Just wanted to ask your thoughts on what to use for analysis and reporting of JMeter test runs.
I read the log analysis options on the Wiki: http://wiki.apache.org/jakarta-jmeter/LogAnalysis. I also added an entry about the Hudson continuous integration server's JMeter plug-in.
However, most solutions there are for JMeter reporting "in-the-large" - i.e. they paint the large picture on how the system copes with load.
Is there a JMeter reporting solution that lets you drill down to specific errors generated by specific samplers? For instance, down to this level of detail:
JMeter Testcase ABC.jmx >
Test Run # 12 >
Thread Group 1, Thread #12 >
Iteration 2 >
Sampler XYZ - Timestamp 2010-04-20 09:01:05 >
[ Assertion Failure Messages ]
[ HTTP Response Body ]
JMeter logs assertion failures by default, and can be configured to capture the full sampler response on error, so this should be possible. I just wanted to ask if any out-of-the-box solution implements this degree of detail.
Regards
Sonam
The information contained in this email and any attached files are strictly
private and confidential. This email should be read by the intended addressee
only. If the recipient of this message is not the intended addressee, please
call Corporate Express Australia Limited on +61 2 9335 0555 or Corporate Express
New Zealand Limited on +64 9 271 7600 and promptly delete this email and any
attachments. The intended recipient of this email may only use, reproduce,
disclose or distribute the information contained in this email and any attached
files with Corporate Express' permission. If you are not the intended addressee,
you are strictly prohibited from using, reproducing, disclosing or distributing
the information contained in this email and any attached files. Corporate
Express advises that this email and any attached files should be scanned to
detect viruses. Corporate Express accepts no liability for loss or damage
(whether caused by negligence or not) resulting from the use of any attached
files.
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
RE: Questions on Jmeter reporting?
Posted by Steve Eckhart <St...@wausaufs.com>.
Sonam,
What I have done is to include identifying information in the Name of the Response Assertion. For example: Batch Summary Search Error ${RowNumber}: ${LogonName} - ${LockboxID} - ${StartDate} - ${EndDate}
The variables are entries in the CSV data files used to create the test. Then, I can go back and try to recreate the error in a browser. This lets me know whether there is a problem in the application (the error appears in a browser, too), in the script not generating the request correctly, or if the error was caused by the load.
Steve Eckhart
Business Analyst/Project Manager
WAUSAU Financial Systems
14010 FNB PKWY STE500
Omaha, NE 68154-5206
Enterprise Payment Solutions.
CONFIDENTIALITY NOTICE: This electronic transmission (including files attached hereto) is intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any disclosure, dissemination, copying, distribution or taking of any action in reliance on the contents of this confidential information is strictly prohibited. If you have received this communication in error, please destroy it and immediately notify us by return email. Thank you.
-----Original Message-----
From: Sonam Chauhan [mailto:sonam.chauhan@ce.com.au]
Sent: Tuesday, April 27, 2010 10:57 PM
To: JMeter Users List
Subject: Questions on Jmeter reporting?
Hello all:
Just wanted to ask your thoughts on what to use for analysis and reporting of JMeter test runs.
I read the log analysis options on the Wiki: http://wiki.apache.org/jakarta-jmeter/LogAnalysis. I also added an entry about the Hudson continuous integration server's JMeter plug-in.
However, most solutions there are for JMeter reporting "in-the-large" - i.e. they paint the large picture on how the system copes with load.
Is there a JMeter reporting solution that lets you drill down to specific errors generated by specific samplers? For instance, down to this level of detail:
JMeter Testcase ABC.jmx >
Test Run # 12 >
Thread Group 1, Thread #12 >
Iteration 2 >
Sampler XYZ - Timestamp 2010-04-20 09:01:05 >
[ Assertion Failure Messages ]
[ HTTP Response Body ]
JMeter logs assertion failures by default, and can be configured to capture the full sampler response on error, so this should be possible. I just wanted to ask if any out-of-the-box solution implements this degree of detail.
Regards
Sonam
The information contained in this email and any attached files are strictly
private and confidential. This email should be read by the intended addressee
only. If the recipient of this message is not the intended addressee, please
call Corporate Express Australia Limited on +61 2 9335 0555 or Corporate Express
New Zealand Limited on +64 9 271 7600 and promptly delete this email and any
attachments. The intended recipient of this email may only use, reproduce,
disclose or distribute the information contained in this email and any attached
files with Corporate Express' permission. If you are not the intended addressee,
you are strictly prohibited from using, reproducing, disclosing or distributing
the information contained in this email and any attached files. Corporate
Express advises that this email and any attached files should be scanned to
detect viruses. Corporate Express accepts no liability for loss or damage
(whether caused by negligence or not) resulting from the use of any attached
files.
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org