You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@jmeter.apache.org by bu...@apache.org on 2016/02/09 20:47:20 UTC
[Bug 58987] New: Report/Dashboard error reporting is hard to follow
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
Bug ID: 58987
Summary: Report/Dashboard error reporting is hard to follow
Product: JMeter
Version: 2.13
Hardware: PC
OS: Mac OS X 10.1
Status: NEW
Severity: normal
Priority: P2
Component: Main
Assignee: issues@jmeter.apache.org
Reporter: sebb@apache.org
I have two very similar-looking CSV files.
One is processed OK, the other generates lots of stack traces.
The difference is that one is missing a column, however this is extremely hard
to determine this from the log file.
There are several issues here:
The error message should explain what the code was trying to do in terms of the
field name(s) it is having problems with. The stack trace is not really helpful
and should be dropped. It would probably help to print the error message(s) to
standard out as well as the log file.
The sample number is shown as #0. It's not clear if that is supposed to be
zero-based or if the count is not being incremented at all.
Since the reporter relies on being able to parse the CSV, and the CSV file
format is not fixed, it seems to me that the solution is to insist on having
the field names in the file. There's no way of knowing whether the CSV file
corresponds with the current property settings - the most that one can hope to
check is whether the number of columns in the CSV agrees with the settings.
The code does not even do that.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard: Improve error reporting
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
Sebb <se...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|RESOLVED |REOPENED
Resolution|FIXED |---
--- Comment #17 from Sebb <se...@apache.org> ---
The file bad.jtl is handled fine.
However, If I run the following test:
./jmeter -n -g ok.jtl -J
jmeter.save.saveservice.assertion_results_failure_message=false
I get the following console message:
An error occurred: Error while processing samples:Consumer failed with message
:Consumer failed with message :Consumer failed with message :Error in sample at
line:1 converting field:bytes at column:8 to:int, fieldValue:''
This is confusing.
Also I would expect to see a column mismatch error, not a parsing error.
Also the jmeter.log file is filled with unnecessary stack traces.
There is no point in providing stack traces here; it's likely to confuse users.
We just need to ensure that the error message is clear.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard: Improve error reporting
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
Philippe Mouawad <p....@ubik-ingenierie.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
--- Comment #13 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
@sebb, is it OK for you ?
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard: Improve error reporting
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
Philippe Mouawad <p....@ubik-ingenierie.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Summary|Report/Dashboard error |Report/Dashboard: Improve
|reporting is hard to follow |error reporting
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard: Improve error reporting
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
--- Comment #18 from Sebb <se...@apache.org> ---
Created attachment 33550
--> https://bz.apache.org/bugzilla/attachment.cgi?id=33550&action=edit
Copy of ok.jtl with last column not a number
This should generate a parse error.
It does, but the message is tricky to read because of all the unnecessary text:
An error occurred: Error while processing samples:Consumer failed with message
:Consumer failed with message :Consumer failed with message :Consumer failed
with message :Error in sample at line:1 converting field:Latency at column:12
to:long, fieldValue:'O'
Also there is no need for a stack trace.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard: Improve error reporting
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
--- Comment #12 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
Author: pmouawad
Date: Wed Feb 10 22:04:30 2016
New Revision: 1729747
URL: http://svn.apache.org/viewvc?rev=1729747&view=rev
Log:
Bug 58987 - Report/Dashboard: Improve error reporting
Bugzilla Id: 58987
Modified:
jmeter/trunk/src/core/org/apache/jmeter/report/core/CsvSampleReader.java
jmeter/trunk/src/core/org/apache/jmeter/report/core/Sample.java
jmeter/trunk/src/core/org/apache/jmeter/report/processor/FieldSampleComparator.java
jmeter/trunk/xdocs/changes.xml
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard: Improve error reporting
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
--- Comment #15 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
(In reply to Sebb from comment #14)
> The line number and field name are now reported.
> However the column number is not reported,
Done
and there are two sets of stack
> traces about the same error.
Fixed
>
> I would expect to see something like:
>
> 2016/02/10 23:53:55 ERROR - jmeter.report.dashboard.ReportGenerator: Error
> in sample at line: 1 converting field <n>: Latency to:long, fieldValue: ''
>
> *without* the other 100 or so lines of stack trace.
I think stacktrace is important for bug reporting.
>
> Also, the code should check whether the column count is correct.
> There's no point trying to parse the data if the count is wrong.
Fixed
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard error reporting is hard to follow
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
--- Comment #3 from Sebb <se...@apache.org> ---
Created attachment 33541
--> https://bz.apache.org/bugzilla/attachment.cgi?id=33541&action=edit
This does not
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard error reporting is hard to follow
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
Philippe Mouawad <p....@ubik-ingenierie.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
CC| |p.mouawad@ubik-ingenierie.c
| |om
--- Comment #1 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
Did you run your test on last nightly build ?
Can you provide a sample and the config of user.properties used in your test ?
Thanks
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard error reporting is hard to follow
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
--- Comment #9 from Sebb <se...@apache.org> ---
(In reply to Philippe Mouawad from comment #8)
> (In reply to Sebb from comment #7)
> > (In reply to Philippe Mouawad from comment #6)
> > > Did you face the issue with a CSV generated with current default settings ?
> > > Thanks
> >
> > No, but that's not the issue.
> >
> > The problem is that generating a report for an existing CSV relies on it
> > having been created with the same properties. That is not guaranteed,
> > especially if the defaults are changed. And users can change the values. And
> > if the CSV was created by a GUI listener, it may not have the same settings.
>
> This would be nice to fix indeed but be aware that it is for example a
> limitation for example of JMeter-Plugins and I wonder if all current
> listeners in JMeter do not have this issue.
No idea what you mean. What limitation are you referring to?
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard error reporting is hard to follow
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
--- Comment #7 from Sebb <se...@apache.org> ---
(In reply to Philippe Mouawad from comment #6)
> Did you face the issue with a CSV generated with current default settings ?
> Thanks
No, but that's not the issue.
The problem is that generating a report for an existing CSV relies on it having
been created with the same properties. That is not guaranteed, especially if
the defaults are changed. And users can change the values. And if the CSV was
created by a GUI listener, it may not have the same settings.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard: Improve error reporting
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
--- Comment #19 from Sebb <se...@apache.org> ---
URL: http://svn.apache.org/viewvc?rev=1730253&view=rev
Log:
Report/Dashboard: Improve error reporting
Throw exception if mismatch between expected number of columns and columns in
csv file (don't allow missing columns)
Bugzilla Id: 58987
Modified:
jmeter/trunk/src/core/org/apache/jmeter/report/core/CsvSampleReader.java
URL: http://svn.apache.org/viewvc?rev=1730255&view=rev
Log:
Revert r1730205 as it results in duplicate information in the log file
Bugzilla Id: 58987
Modified:
jmeter/trunk/src/core/org/apache/jmeter/JMeter.java
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard: Improve error reporting
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
Philippe Mouawad <p....@ubik-ingenierie.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |FIXED
--- Comment #16 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
Author: pmouawad
Date: Sat Feb 13 13:43:44 2016
New Revision: 1730205
URL: http://svn.apache.org/viewvc?rev=1730205&view=rev
Log:
Bug 58987 - Report/Dashboard: Improve error reporting
Avoid Log+Rethrow, only rethrow
Add column index in message
Bugzilla Id: 58987
Modified:
jmeter/trunk/src/core/org/apache/jmeter/JMeter.java
jmeter/trunk/src/core/org/apache/jmeter/report/core/Sample.java
jmeter/trunk/src/core/org/apache/jmeter/report/dashboard/ReportGenerator.java
jmeter/trunk/src/core/org/apache/jmeter/report/processor/AbstractSampleConsumer.java
jmeter/trunk/src/core/org/apache/jmeter/report/processor/CsvFileSampleSource.java
Author: pmouawad
Date: Sat Feb 13 13:53:46 2016
New Revision: 1730206
URL: http://svn.apache.org/viewvc?rev=1730206&view=rev
Log:
Bug 58987 - Report/Dashboard: Improve error reporting
Throw exception if mismatch between expected number of columns and columns in
csv file
Bugzilla Id: 58987
Modified:
jmeter/trunk/src/core/org/apache/jmeter/report/core/CsvSampleReader.java
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard: Improve error reporting
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
Philippe Mouawad <p....@ubik-ingenierie.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|REOPENED |RESOLVED
Resolution|--- |FIXED
--- Comment #20 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
Improved also with commits:
http://svn.apache.org/viewvc?rev=1731954&view=rev
http://svn.apache.org/viewvc?rev=1731945&view=rev
http://svn.apache.org/viewvc?rev=1731944&view=rev
Closing.
Feel free to reopen if not enough
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard error reporting is hard to follow
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
Sebb <se...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEEDINFO |NEW
--- Comment #5 from Sebb <se...@apache.org> ---
I used current trunk freshly built. No changes to jmeter.properties
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard: Improve error reporting
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
Sebb <se...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEEDINFO |NEW
--- Comment #14 from Sebb <se...@apache.org> ---
The line number and field name are now reported.
However the column number is not reported, and there are two sets of stack
traces about the same error.
I would expect to see something like:
2016/02/10 23:53:55 ERROR - jmeter.report.dashboard.ReportGenerator: Error in
sample at line: 1 converting field <n>: Latency to:long, fieldValue: ''
*without* the other 100 or so lines of stack trace.
Also, the code should check whether the column count is correct.
There's no point trying to parse the data if the count is wrong.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard error reporting is hard to follow
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
Philippe Mouawad <p....@ubik-ingenierie.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
OS|Mac OS X 10.1 |All
Hardware|PC |All
Severity|normal |enhancement
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard error reporting is hard to follow
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
--- Comment #2 from Sebb <se...@apache.org> ---
Created attachment 33540
--> https://bz.apache.org/bugzilla/attachment.cgi?id=33540&action=edit
This parses OK
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard error reporting is hard to follow
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
--- Comment #4 from Sebb <se...@apache.org> ---
Created attachment 33542
--> https://bz.apache.org/bugzilla/attachment.cgi?id=33542&action=edit
Sample log output
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard error reporting is hard to follow
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
--- Comment #6 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
Did you face the issue with a CSV generated with current default settings ?
Thanks
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard error reporting is hard to follow
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
--- Comment #10 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
With jmeter-plugins, if you try to generate graphs from a CSV file generated
with different jmeter properties, it will also fail.
You need to have the same properties that led to the CSV file.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard error reporting is hard to follow
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
--- Comment #11 from Sebb <se...@apache.org> ---
(In reply to Philippe Mouawad from comment #10)
> With jmeter-plugins, if you try to generate graphs from a CSV file generated
> with different jmeter properties, it will also fail.
> You need to have the same properties that led to the CSV file.
I see. This is the same with JMeter's Listeners.
But that does not mean that nothing can or should be done to improve the
behaviour of report dashboard. It's particularly important for this listener as
it is new, and so users will want to apply it to existing CSV files.
So we should:
- log/print a warning if there are no field names
- if no field names, then check that the number fields agrees with the
properties, and quit if there is a mismatch
- if there is a parsing error, report the record, field number, and expected
field name to enable the problem to be debugged.
Some of these improvements can also be applied to the other JMeter listeners.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 58987] Report/Dashboard error reporting is hard to follow
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58987
--- Comment #8 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
(In reply to Sebb from comment #7)
> (In reply to Philippe Mouawad from comment #6)
> > Did you face the issue with a CSV generated with current default settings ?
> > Thanks
>
> No, but that's not the issue.
>
> The problem is that generating a report for an existing CSV relies on it
> having been created with the same properties. That is not guaranteed,
> especially if the defaults are changed. And users can change the values. And
> if the CSV was created by a GUI listener, it may not have the same settings.
This would be nice to fix indeed but be aware that it is for example a
limitation for example of JMeter-Plugins and I wonder if all current listeners
in JMeter do not have this issue.
--
You are receiving this mail because:
You are the assignee for the bug.