You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@jmeter.apache.org by bu...@apache.org on 2016/03/31 23:08:58 UTC
[Bug 59258] New: GUI Mode : OOM Protection for View Results Tree or
View Results in Table
https://bz.apache.org/bugzilla/show_bug.cgi?id=59258
Bug ID: 59258
Summary: GUI Mode : OOM Protection for View Results Tree or
View Results in Table
Product: JMeter
Version: 2.13
Hardware: All
OS: All
Status: NEW
Severity: enhancement
Priority: P2
Component: Main
Assignee: issues@jmeter.apache.org
Reporter: p.mouawad@ubik-ingenierie.com
As you know, unfortunately and despite all the warnings and best practices we
wrote, a lot of users still use GUI Mode when Load Testing.
This is an issue when Listener such as View Results Tree or View Results in
Table are used.
This ends usually with "abnormal and wrong" slow response times before an OOM
occurs.
This issue is one of the major factor for JMeter being felt as "unstable" I
think.
I propose the following solution that I have already developed
Add 2 user properties:
#view.results.table.max_rows=5000
#view.results.tree.max_nodes=5000
In the "View Results Tree " , when number of nodes in the tree reaches the max,
we remove the first node in the Tree.
In the "View Results in Table" , when number of rows in the table reaches the
max, we remove the first row in the table.
So we limit this way the number of nodes and rows in those listeners.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 59258] GUI Mode : OOM Protection for View Results Tree or View
Results in Table
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59258
--- Comment #15 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
(In reply to Sebb from comment #11)
> (In reply to Philippe Mouawad from comment #10)
> > (In reply to Sebb from comment #9)
> > > (In reply to Philippe Mouawad from comment #8)
> > > > Hi sebb,
> > > > Just to illustrate, first question now on stackoverflow:
> > > > http://stackoverflow.com/questions/tagged/jmeter
> > > >
> > > > Jmeter freezes with a CPU of 100%
> > > >
> > > >
> > > > Search in it and on our mailing list, I bet you give tens of entries
> > >
> > > Yes, there probably are.
> > > There are lots of entries about other common problems which would be solved
> > > if people read the manual.
> >
> > Yes, but if we can make it OOTB then why not
>
> Because of the disadvantages.
>
> > >
> > > I agree that if we can make a simple change to JMeter that makes it easier
> > > to use and does not compromise performance then we should consider doing it.
> > >
> > > However in this case any changes are likely to be intrusive, so the
> > > trade-off is not worth it.
> >
> > I don't share this opinion.
>
> Are you saying that the changes are not intrusive, or that even though they
> are intrusive, it's worth the disadvantages?
Even if they are intrusive it's worth the disadvantage.
Without protection : Results are KO and JMeter crashes
With protection : Results are KO, user is informed and JMeter does not crash
>
> > >
> > > Also, the test will still reach saturation earlier than it would do if they
> > > had not used the Listener, so they are then going to complain that JMeter
> > > does not scale.
> >
> > But that won't be true.
>
> On the contrary, it is true that using one of these Listeners will reduce
> the max throughput that JMeter can achieve.
> Maybe not in all tests, but it will be important in some.
>
> Using extra storage can affect the max possible throughput.
> Using extra processing to implement the limiting also affects the max
> throughput.
>
> > While currently saying that JMeter does not scale
> > with GUI Mode + View Results Tree is not strictly wrong.
>
> Not sure what that means.
Currently, if someone uses GUI mode with View Results Tree for load testing,
JMeter does not scale. Although this is against best practices we documented,
this is possible.
With protection it is not anymore.
>
> >
> > >I think it's just postponing the problem.
> >
> >
> > As a conclusion , maybe Vladimir has a brilliant idea that could help here.
>
> Maybe.
>
> > If not then I suggest 2 options:
> > - We use my patch and accept drawback
>
> -1
>
> > - We use your option (drop additional sampler) AND add a warning message on
> > GUI to say that this is in place
>
> Even though I suggested it, I don't think it's a viable idea.
I implemented the 2 options , I attached the patch.
Frankly I think it is an improvement.
I am really tired of seeing people fall into this trap too easily.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 59258] GUI Mode : OOM Protection for View Results Tree or View
Results in Table
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59258
--- Comment #8 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
Hi sebb,
Just to illustrate, first question now on stackoverflow:
http://stackoverflow.com/questions/tagged/jmeter
Jmeter freezes with a CPU of 100%
Search in it and on our mailing list, I bet you give tens of entries
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 59258] GUI Mode : OOM Protection for View Results Tree or View
Results in Table
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59258
Philippe Mouawad <p....@ubik-ingenierie.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |DUPLICATE
--- Comment #17 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
*** This bug has been marked as a duplicate of bug 60687 ***
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 59258] GUI Mode : OOM Protection for View Results Tree or View
Results in Table
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59258
--- Comment #11 from Sebb <se...@apache.org> ---
(In reply to Philippe Mouawad from comment #10)
> (In reply to Sebb from comment #9)
> > (In reply to Philippe Mouawad from comment #8)
> > > Hi sebb,
> > > Just to illustrate, first question now on stackoverflow:
> > > http://stackoverflow.com/questions/tagged/jmeter
> > >
> > > Jmeter freezes with a CPU of 100%
> > >
> > >
> > > Search in it and on our mailing list, I bet you give tens of entries
> >
> > Yes, there probably are.
> > There are lots of entries about other common problems which would be solved
> > if people read the manual.
>
> Yes, but if we can make it OOTB then why not
Because of the disadvantages.
> >
> > I agree that if we can make a simple change to JMeter that makes it easier
> > to use and does not compromise performance then we should consider doing it.
> >
> > However in this case any changes are likely to be intrusive, so the
> > trade-off is not worth it.
>
> I don't share this opinion.
Are you saying that the changes are not intrusive, or that even though they are
intrusive, it's worth the disadvantages?
> >
> > Also, the test will still reach saturation earlier than it would do if they
> > had not used the Listener, so they are then going to complain that JMeter
> > does not scale.
>
> But that won't be true.
On the contrary, it is true that using one of these Listeners will reduce the
max throughput that JMeter can achieve.
Maybe not in all tests, but it will be important in some.
Using extra storage can affect the max possible throughput.
Using extra processing to implement the limiting also affects the max
throughput.
> While currently saying that JMeter does not scale
> with GUI Mode + View Results Tree is not strictly wrong.
Not sure what that means.
>
> >I think it's just postponing the problem.
>
>
> As a conclusion , maybe Vladimir has a brilliant idea that could help here.
Maybe.
> If not then I suggest 2 options:
> - We use my patch and accept drawback
-1
> - We use your option (drop additional sampler) AND add a warning message on
> GUI to say that this is in place
Even though I suggested it, I don't think it's a viable idea.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 59258] GUI Mode : OOM Protection for View Results Tree or View
Results in Table
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59258
Philippe Mouawad <p....@ubik-ingenierie.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |p.mouawad@ubik-ingenierie.c
| |om
--- Comment #1 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
Created attachment 33716
--> https://bz.apache.org/bugzilla/attachment.cgi?id=33716&action=edit
Patch for the issue
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 59258] GUI Mode : OOM Protection for View Results Tree or View
Results in Table
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59258
--- Comment #5 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
(In reply to Sebb from comment #4)
> I think Vladimir means that removal of the first element of those data
> structures is an expensive operation.
Ok I agree
>
> Therefore the 'cure' might cause other problems.
>
> As to the solution, my preference is to not try to fix it through code.
What do you propose then ?
>
> However of course it would be quite cheap to start dropping entries when a
> limit is reached.
Isn't it what my patch does ?
If it's cheap , then why not do it this way ?
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 59258] GUI Mode : OOM Protection for View Results Tree or View
Results in Table
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59258
--- Comment #9 from Sebb <se...@apache.org> ---
(In reply to Philippe Mouawad from comment #8)
> Hi sebb,
> Just to illustrate, first question now on stackoverflow:
> http://stackoverflow.com/questions/tagged/jmeter
>
> Jmeter freezes with a CPU of 100%
>
>
> Search in it and on our mailing list, I bet you give tens of entries
Yes, there probably are.
There are lots of entries about other common problems which would be solved if
people read the manual.
I agree that if we can make a simple change to JMeter that makes it easier to
use and does not compromise performance then we should consider doing it.
However in this case any changes are likely to be intrusive, so the trade-off
is not worth it.
Also, the test will still reach saturation earlier than it would do if they had
not used the Listener, so they are then going to complain that JMeter does not
scale. I think it's just postponing the problem.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 59258] GUI Mode : OOM Protection for View Results Tree or View
Results in Table
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59258
--- Comment #16 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
This PR (https://github.com/apache/jmeter/pull/179) is a first try to base
protection on memory estimation.
Memory estimation of SampleResult is too positive I think, it does only take
into account the AssertionResult and SampleResult response data, not the
properties size. This could be improved.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 59258] GUI Mode : OOM Protection for View Results Tree or View
Results in Table
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59258
--- Comment #6 from Sebb <se...@apache.org> ---
(In reply to Philippe Mouawad from comment #5)
> (In reply to Sebb from comment #4)
> > I think Vladimir means that removal of the first element of those data
> > structures is an expensive operation.
>
> Ok I agree
>
> >
> > Therefore the 'cure' might cause other problems.
> >
> > As to the solution, my preference is to not try to fix it through code.
>
> What do you propose then ?
Continue with education.
> >
> > However of course it would be quite cheap to start dropping entries when a
> > limit is reached.
>
> Isn't it what my patch does ?
> If it's cheap , then why not do it this way ?
Sorry, meant to write 'samples' not 'entries' - i.e. don't add another entry if
the list is full. That should be cheap and solves the OOM.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 59258] GUI Mode : OOM Protection for View Results Tree or View
Results in Table
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59258
--- Comment #4 from Sebb <se...@apache.org> ---
I think Vladimir means that removal of the first element of those data
structures is an expensive operation.
Therefore the 'cure' might cause other problems.
As to the solution, my preference is to not try to fix it through code.
However of course it would be quite cheap to start dropping entries when a
limit is reached.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 59258] GUI Mode : OOM Protection for View Results Tree or View
Results in Table
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59258
--- Comment #10 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
(In reply to Sebb from comment #9)
> (In reply to Philippe Mouawad from comment #8)
> > Hi sebb,
> > Just to illustrate, first question now on stackoverflow:
> > http://stackoverflow.com/questions/tagged/jmeter
> >
> > Jmeter freezes with a CPU of 100%
> >
> >
> > Search in it and on our mailing list, I bet you give tens of entries
>
> Yes, there probably are.
> There are lots of entries about other common problems which would be solved
> if people read the manual.
Yes, but if we can make it OOTB then why not
>
> I agree that if we can make a simple change to JMeter that makes it easier
> to use and does not compromise performance then we should consider doing it.
>
> However in this case any changes are likely to be intrusive, so the
> trade-off is not worth it.
I don't share this opinion.
>
> Also, the test will still reach saturation earlier than it would do if they
> had not used the Listener, so they are then going to complain that JMeter
> does not scale.
But that won't be true. While currently saying that JMeter does not scale with
GUI Mode + View Results Tree is not strictly wrong.
>I think it's just postponing the problem.
As a conclusion , maybe Vladimir has a brilliant idea that could help here.
If not then I suggest 2 options:
- We use my patch and accept drawback
- We use your option (drop additional sampler) AND add a warning message on GUI
to say that this is in place
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 59258] GUI Mode : OOM Protection for View Results Tree or View
Results in Table
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59258
--- Comment #7 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
(In reply to Sebb from comment #6)
> (In reply to Philippe Mouawad from comment #5)
> > (In reply to Sebb from comment #4)
> > > I think Vladimir means that removal of the first element of those data
> > > structures is an expensive operation.
> >
> > Ok I agree
> >
> > >
> > > Therefore the 'cure' might cause other problems.
> > >
> > > As to the solution, my preference is to not try to fix it through code.
> >
> > What do you propose then ?
>
> Continue with education.
Unfortunately, I am not sure it's enough.
>
> > >
> > > However of course it would be quite cheap to start dropping entries when a
> > > limit is reached.
> >
> > Isn't it what my patch does ?
> > If it's cheap , then why not do it this way ?
>
> Sorry, meant to write 'samples' not 'entries' - i.e. don't add another entry
> if the list is full. That should be cheap and solves the OOM.
Yes that's another way to fix it.
But if you take a long running test:
- I suppose user would be more interested in last samples than old ones
- He might think JMeter is stuck, which ends up the same from User perception :
"JMeter is unstable, my test is stuck ...."
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 59258] GUI Mode : OOM Protection for View Results Tree or View
Results in Table
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59258
--- Comment #3 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
(In reply to Vladimir Sitnikov from comment #2)
> Philippe, are you aware that both removals are "remove the first element out
> of arraylist"? Those seem to be pathological cases.
Can you clarify what you mean ?
>
> ObjectTableModel stores model in ArrayList, while DefaultTreeModel stores
> model in Vector.
>
> Have you tested which overhead creates that node removal?
No.
As for me anyway Load Testing in GUI mode is evil.
I am just here protecting JMeter from OOM
> I'm not fond of running long operations under synchronization.
What other solution do we have ?
Do this only protection part every N Inserts ?
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 59258] GUI Mode : OOM Protection for View Results Tree or View
Results in Table
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59258
--- Comment #14 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
Created attachment 33720
--> https://bz.apache.org/bugzilla/attachment.cgi?id=33720&action=edit
Screenshot showing the warning on ViewResultsTable
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 59258] GUI Mode : OOM Protection for View Results Tree or View
Results in Table
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59258
--- Comment #12 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
Created attachment 33718
--> https://bz.apache.org/bugzilla/attachment.cgi?id=33718&action=edit
Patch that does not add SampleResults anymore and displays a warning
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 59258] GUI Mode : OOM Protection for View Results Tree or View
Results in Table
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59258
--- Comment #2 from Vladimir Sitnikov <si...@gmail.com> ---
Philippe, are you aware that both removals are "remove the first element out of
arraylist"? Those seem to be pathological cases.
ObjectTableModel stores model in ArrayList, while DefaultTreeModel stores model
in Vector.
Have you tested which overhead creates that node removal?
I'm not fond of running long operations under synchronization.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 59258] GUI Mode : OOM Protection for View Results Tree or View
Results in Table
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59258
--- Comment #13 from Philippe Mouawad <p....@ubik-ingenierie.com> ---
Created attachment 33719
--> https://bz.apache.org/bugzilla/attachment.cgi?id=33719&action=edit
Screenshot showing the warning on ViewResultsTree
--
You are receiving this mail because:
You are the assignee for the bug.