You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jmeter.apache.org by Philippe Mouawad <ph...@gmail.com> on 2017/02/02 19:50:08 UTC
Re: GUI Mode : OOM Protection for View Results Tree or View Results
in Table
Hello,
Despite all the work we've done these last years to warn users about not
using JMeter in GUI mode with load testing, it appears this mode continues
to hurt JMeter's stability reputation.
For example, you can see in this 2016 comparison of tools some highlights:
http://okt.to/6aA0hS
We're among the best but I am personally fed up of still reading those
reviews that mention freezes of GUI under load.
We discussed previously about finding a solution to it but didn't implement
anything yet.
Maybe it's time to tackle and kill this subject.
I add one proposal:
- By default have a property that limits the number of Threads that can
run in GUI mode. We could dynamically based on number of Cores and size of
Heap compute a max value. Then during the building of Tree for test with a
TreeCloner, JMeter would either :
- refuse to launch test plan and popup a warn to user telling him to
either reduce number of VUs or run in Non GUI mode.
- block number of VU to the max.
- Users could switch off this by settings the property that would be
explained in popup.
Other thoughts ?
Thanks
Regards
Philippe M.
On Sun, Apr 3, 2016 at 6:04 PM, Philippe Mouawad <philippe.mouawad@gmail.com
> wrote:
> ok for me
>
>
> On Sunday, April 3, 2016, Felix Schumacher <felix.schumacher@
> internetallee.de> wrote:
>
>>
>>
>> Am 3. April 2016 14:41:56 MESZ, schrieb Philippe Mouawad <
>> philippe.mouawad@gmail.com>:
>> >Hi Vladimir,
>> >I submitted a PR .
>> >You can improve it.
>>
>> Do we have to rush here?
>>
>> Why not wait till after the release.
>>
>> Regards,
>> Felix
>>
>> >Thanks
>> >
>> >On Sat, Apr 2, 2016 at 11:12 PM, Vladimir Sitnikov <
>> >sitnikov.vladimir@gmail.com> wrote:
>> >
>> >> > or approximate the size in bytes of Nodes / Rows ?
>> >>
>> >> This one.
>> >>
>> >> I think it should not be that hard to estimate SampleResult size in
>> >bytes
>> >> for
>> >> common implementations. Same for node in result tree/result table.
>> >> So we can implement user-facing property like "stop adding more
>> >> results if existing occupy 50MiB".
>> >>
>> >> We can keep "SampleResult.estimateSize()" API as JMeter-core-BETA-API
>> >> (e.g. not to be used by extensions) so we can remove that if the
>> >approach
>> >> turns
>> >> out to be not that good.
>> >>
>> >> Vladimir
>> >>
>>
>>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: GUI Mode : OOM Protection for View Results Tree or View Results
in Table
Posted by Vladimir Sitnikov <si...@gmail.com>.
Felix>You mean as a guard before we use isEventDispatchThread?
Exactly. It would be good if there are other ways to tell if code is
running "in the event dispatch thread", however I can't find that in the
awt api, so it has to be something tied to JMeter. "instanceof
JMeterThread" was just the first thing that came to my mind.
Vladimir> For invokeLater I think we can cache
Vladimir> Toolkit.getDefaultToolit().getSystemEventQueue() and then use
Vladimir> eventQueue.postEvent(new InvocationEvent(toolkit, runnable)).
Felix>Is that a common way to optimize awt programs?
To be honest, I've very little experience with swing/awt/whatever ui
programming in java, and those suggestions are based just on my analysis of
the thread dumps.
Vladimir
Re: GUI Mode : OOM Protection for View Results Tree or View Results
in Table
Posted by Felix Schumacher <fe...@internetallee.de>.
Am 03.02.2017 um 16:36 schrieb Vladimir Sitnikov:
> Felix>I tried it out and the resuls are promising.
>
> Just one more finding:
> SwingUtilities.invokeLater and SwingUtilities.isEventDispatchThread involve
> global synchronization (that is java 1.8) as follows:
>
> at sun.awt.AppContext.get(Unknown Source)
> - waiting to lock <0x00000000c0003d30> (a java.util.HashMap)
> at sun.awt.SunToolkit.getSystemEventQueueImplPP(Unknown Source)
> at sun.awt.SunToolkit.getSystemEventQueueImplPP(Unknown Source)
> at sun.awt.SunToolkit.getSystemEventQueueImpl(Unknown Source)
> at java.awt.Toolkit.getEventQueue(Unknown Source)
> at java.awt.EventQueue.isDispatchThread(Unknown Source)
> at javax.swing.SwingUtilities.isEventDispatchThread(Unknown Source)
> at org.apache.jmeter.util.JMeterUtils.runSafe(JMeterUtils.java:1373)
>
>
> It does look like an AWT issue, however we'd better workaround that somehow.
>
> For "isEventDispatchThread" we can use "if )currentThread instanceof
> JMeterThread) { 100% non-event dispatch thread}".
You mean as a guard before we use isEventDispatchThread?
>
> For invokeLater I think we can cache
> Toolkit.getDefaultToolit().getSystemEventQueue() and then use
> eventQueue.postEvent(new InvocationEvent(toolkit, runnable)).
Is that a common way to optimize awt programs?
Regards,
Felix
>
> That should be a trivial change to improve scalability and responsiveness.
>
> Of course, it would still make sense to coalesce events (e.g. drop old log
> records) before they reach awt (I mean avoid lots of eventQueue.postEvent
> calls by buffering the events before they got sent to swing) since awt's
> eventQueue does not scale.
>
> Vladimir
>
Re: GUI Mode : OOM Protection for View Results Tree or View Results
in Table
Posted by Vladimir Sitnikov <si...@gmail.com>.
Felix>I tried it out and the resuls are promising.
Just one more finding:
SwingUtilities.invokeLater and SwingUtilities.isEventDispatchThread involve
global synchronization (that is java 1.8) as follows:
at sun.awt.AppContext.get(Unknown Source)
- waiting to lock <0x00000000c0003d30> (a java.util.HashMap)
at sun.awt.SunToolkit.getSystemEventQueueImplPP(Unknown Source)
at sun.awt.SunToolkit.getSystemEventQueueImplPP(Unknown Source)
at sun.awt.SunToolkit.getSystemEventQueueImpl(Unknown Source)
at java.awt.Toolkit.getEventQueue(Unknown Source)
at java.awt.EventQueue.isDispatchThread(Unknown Source)
at javax.swing.SwingUtilities.isEventDispatchThread(Unknown Source)
at org.apache.jmeter.util.JMeterUtils.runSafe(JMeterUtils.java:1373)
It does look like an AWT issue, however we'd better workaround that somehow.
For "isEventDispatchThread" we can use "if )currentThread instanceof
JMeterThread) { 100% non-event dispatch thread}".
For invokeLater I think we can cache
Toolkit.getDefaultToolit().getSystemEventQueue() and then use
eventQueue.postEvent(new InvocationEvent(toolkit, runnable)).
That should be a trivial change to improve scalability and responsiveness.
Of course, it would still make sense to coalesce events (e.g. drop old log
records) before they reach awt (I mean avoid lots of eventQueue.postEvent
calls by buffering the events before they got sent to swing) since awt's
eventQueue does not scale.
Vladimir
Re: GUI Mode : OOM Protection for View Results Tree or View Results
in Table
Posted by Felix Schumacher <fe...@internetallee.de>.
Am 03.02.2017 08:21, schrieb Felix Schumacher:
> Am 2. Februar 2017 21:44:43 MEZ schrieb Vladimir Sitnikov
> <si...@gmail.com>:
>>> Can you clarify your question if it's a real one.
>>
>> Well, that's a real question.
>> It is a long time since I launched a load test, and I can't remember
>> if
>> faced "UI stuck" error ever.
>>
>> I would admit I've never used non-GUI mode, and GUI was just fine for
>> me
>> (of course I had to disable things like "view results").
>>
>> Philippe>Did you see the last thread "High CPU utilization in
>> JMeter 3.x with HttpClient 4 leads to freeze" ?
>>
>> I think I missed that.
>> A quick glance (e.g.
>> https://drive.google.com/drive/folders/0B1uBdVuLED5NejZaanZ2UHNlblk )
>> shows
>> that we have a logging problem.
>>
>> That is org.apache.jmeter.gui.LoggerPanel.processEvent might fill UI
>> event
>> queue, thus rendering UI unresponsive.
>>
>> We might want to have some throttling there, or circular buffering, or
>> lazy-loading (e.g. logging to a file and refreshing it to the UI from
>> time
>> to time).
>>
>> In any way, the number of threads itself has nothing to do with UI
>> responsiveness provided there is enough Xmx (remember we did discuss
>> "please specify new Xmx value and click restart" dialog?) and there is
>> enough CPU power (in case of CPU starvation the load test makes no
>> sense at
>> all).
>>
>> Philippe>blocks Load test mode
>>
>> You are right. It is much better to have application level error that
>> explains what goes wrong rather than fail with some obscure error at
>> runtime.
>>
>> However I would like to explore if we can have a way out without
>> stopping
>> the show.
>>
>> When captain opens the thread dump above, he would find some obvious
>> things:
>> 1) Logging fills the UI queue. The simplest workaround would be to
>> disable
>> log panel updates after N messages.
>> 2) Statistics posts UI events as well (SummaryReport.add). Can we have
>> some
>> debounce there, so the UI gets refreshed once a second at most?
>>
>> Well, the above do not look like a quick win, however I think those
>> can
>> be
>> fixed without complicating the codebase too much.
>>
>> Does it make sense to spawn a couple of new mail threads regarding
>> those
>> issues?
>
> I think those are valid things to try out.
I tried it out and the resuls are promising.
Usage (%) in AWT Dispatcher
old code: 88
new (100ms): 44
new (500ms): 14
Usage (%) in Thread (mostly the JSR223-Sampler)
old code: 11
new (100ms): 55
new (500ms): 84
The test case I used was one Thread Group with 100 Threads and a single
JSR223-Sampler with "log.info("Blubb");"
Memory Usage (speak GC) went down to.
Felix
>
> Felix
>
>>
>> Vladimir
Re: GUI Mode : OOM Protection for View Results Tree or View Results in Table
Posted by Felix Schumacher <fe...@internetallee.de>.
Am 2. Februar 2017 21:44:43 MEZ schrieb Vladimir Sitnikov <si...@gmail.com>:
>>Can you clarify your question if it's a real one.
>
>Well, that's a real question.
>It is a long time since I launched a load test, and I can't remember if
>faced "UI stuck" error ever.
>
>I would admit I've never used non-GUI mode, and GUI was just fine for
>me
>(of course I had to disable things like "view results").
>
>Philippe>Did you see the last thread "High CPU utilization in
>JMeter 3.x with HttpClient 4 leads to freeze" ?
>
>I think I missed that.
>A quick glance (e.g.
>https://drive.google.com/drive/folders/0B1uBdVuLED5NejZaanZ2UHNlblk )
>shows
>that we have a logging problem.
>
>That is org.apache.jmeter.gui.LoggerPanel.processEvent might fill UI
>event
>queue, thus rendering UI unresponsive.
>
>We might want to have some throttling there, or circular buffering, or
>lazy-loading (e.g. logging to a file and refreshing it to the UI from
>time
>to time).
>
>In any way, the number of threads itself has nothing to do with UI
>responsiveness provided there is enough Xmx (remember we did discuss
>"please specify new Xmx value and click restart" dialog?) and there is
>enough CPU power (in case of CPU starvation the load test makes no
>sense at
>all).
>
>Philippe>blocks Load test mode
>
>You are right. It is much better to have application level error that
>explains what goes wrong rather than fail with some obscure error at
>runtime.
>
>However I would like to explore if we can have a way out without
>stopping
>the show.
>
>When captain opens the thread dump above, he would find some obvious
>things:
>1) Logging fills the UI queue. The simplest workaround would be to
>disable
>log panel updates after N messages.
>2) Statistics posts UI events as well (SummaryReport.add). Can we have
>some
>debounce there, so the UI gets refreshed once a second at most?
>
>Well, the above do not look like a quick win, however I think those can
>be
>fixed without complicating the codebase too much.
>
>Does it make sense to spawn a couple of new mail threads regarding
>those
>issues?
I think those are valid things to try out.
Felix
>
>Vladimir
Re: GUI Mode : OOM Protection for View Results Tree or View Results
in Table
Posted by Philippe Mouawad <ph...@gmail.com>.
On Thu, Feb 2, 2017 at 11:12 PM, Vladimir Sitnikov <
sitnikov.vladimir@gmail.com> wrote:
> Philippe>For me there is jmeter.loggerpanel.maxlength which does not allow
> settings
> Philippe>more than 80k chars no ?
>
> I saw the bug, however can you please answer the following question:
> do you really intend to update log panel for each and single update from
> every log message appearing in load threads?
>
I don't intend, here is existing behaviour:
https://github.com/apache/jmeter/blob/trunk/src/core/org/apache/jmeter/gui/LoggerPanel.java#L111
>
> I don't think the answer depends on the presence or absence of the bug.
>
> Philippe>I'd prefer a couple of PR :-)
>
> Whatever gets PRed gets merged. Got the idea. )
>
> Vladimir
>
--
Cordialement.
Philippe Mouawad.
Re: GUI Mode : OOM Protection for View Results Tree or View Results
in Table
Posted by Vladimir Sitnikov <si...@gmail.com>.
Philippe>For me there is jmeter.loggerpanel.maxlength which does not allow
settings
Philippe>more than 80k chars no ?
I saw the bug, however can you please answer the following question:
do you really intend to update log panel for each and single update from
every log message appearing in load threads?
I don't think the answer depends on the presence or absence of the bug.
Philippe>I'd prefer a couple of PR :-)
Whatever gets PRed gets merged. Got the idea. )
Vladimir
Re: GUI Mode : OOM Protection for View Results Tree or View Results
in Table
Posted by Philippe Mouawad <ph...@gmail.com>.
On Thu, Feb 2, 2017 at 9:44 PM, Vladimir Sitnikov <
sitnikov.vladimir@gmail.com> wrote:
> >Can you clarify your question if it's a real one.
>
> Well, that's a real question.
> It is a long time since I launched a load test, and I can't remember if
> faced "UI stuck" error ever.
>
> I would admit I've never used non-GUI mode, and GUI was just fine for me
> (of course I had to disable things like "view results").
>
Are you sure that for example using Summary report won't lead to freezes ?
I wouldn't bet it.
>
> Philippe>Did you see the last thread "High CPU utilization in
> JMeter 3.x with HttpClient 4 leads to freeze" ?
>
> I think I missed that.
> A quick glance (e.g.
> https://drive.google.com/drive/folders/0B1uBdVuLED5NejZaanZ2UHNlblk )
> shows
> that we have a logging problem.
>
> That is org.apache.jmeter.gui.LoggerPanel.processEvent might fill UI
> event
> queue, thus rendering UI unresponsive.
>
> We might want to have some throttling there, or circular buffering, or
> lazy-loading (e.g. logging to a file and refreshing it to the UI from time
> to time).
>
For me there is jmeter.loggerpanel.maxlength which does not allow settings
more than 80k chars no ?
>
> In any way, the number of threads itself has nothing to do with UI
> responsiveness provided there is enough Xmx (remember we did discuss
> "please specify new Xmx value and click restart" dialog?) and there is
> enough CPU power (in case of CPU starvation the load test makes no sense at
> all).
>
That's definitely the issue Vladimir.
Many newbies or not aware of that.
>
> Philippe>blocks Load test mode
>
> You are right. It is much better to have application level error that
> explains what goes wrong rather than fail with some obscure error at
> runtime.
>
> However I would like to explore if we can have a way out without stopping
> the show.
>
I'm proposing to not start it :-)
>
> When captain opens the thread dump above, he would find some obvious
> things:
> 1) Logging fills the UI queue. The simplest workaround would be to disable
> log panel updates after N messages.
>
See above but I think calling setText with big text has horrid performances
as per the JDK bug opened recently.
> 2) Statistics posts UI events as well (SummaryReport.add). Can we have some
> debounce there, so the UI gets refreshed once a second at most?
>
Why not
>
> Well, the above do not look like a quick win, however I think those can be
> fixed without complicating the codebase too much.
>
> Does it make sense to spawn a couple of new mail threads regarding those
> issues?
>
I'd prefer a couple of PR :-)
>
> Vladimir
>
--
Cordialement.
Philippe Mouawad.
Re: GUI Mode : OOM Protection for View Results Tree or View Results
in Table
Posted by Vladimir Sitnikov <si...@gmail.com>.
>Can you clarify your question if it's a real one.
Well, that's a real question.
It is a long time since I launched a load test, and I can't remember if
faced "UI stuck" error ever.
I would admit I've never used non-GUI mode, and GUI was just fine for me
(of course I had to disable things like "view results").
Philippe>Did you see the last thread "High CPU utilization in
JMeter 3.x with HttpClient 4 leads to freeze" ?
I think I missed that.
A quick glance (e.g.
https://drive.google.com/drive/folders/0B1uBdVuLED5NejZaanZ2UHNlblk ) shows
that we have a logging problem.
That is org.apache.jmeter.gui.LoggerPanel.processEvent might fill UI event
queue, thus rendering UI unresponsive.
We might want to have some throttling there, or circular buffering, or
lazy-loading (e.g. logging to a file and refreshing it to the UI from time
to time).
In any way, the number of threads itself has nothing to do with UI
responsiveness provided there is enough Xmx (remember we did discuss
"please specify new Xmx value and click restart" dialog?) and there is
enough CPU power (in case of CPU starvation the load test makes no sense at
all).
Philippe>blocks Load test mode
You are right. It is much better to have application level error that
explains what goes wrong rather than fail with some obscure error at
runtime.
However I would like to explore if we can have a way out without stopping
the show.
When captain opens the thread dump above, he would find some obvious things:
1) Logging fills the UI queue. The simplest workaround would be to disable
log panel updates after N messages.
2) Statistics posts UI events as well (SummaryReport.add). Can we have some
debounce there, so the UI gets refreshed once a second at most?
Well, the above do not look like a quick win, however I think those can be
fixed without complicating the codebase too much.
Does it make sense to spawn a couple of new mail threads regarding those
issues?
Vladimir
Re: GUI Mode : OOM Protection for View Results Tree or View Results
in Table
Posted by Philippe Mouawad <ph...@gmail.com>.
On Thu, Feb 2, 2017 at 9:02 PM, Vladimir Sitnikov <
sitnikov.vladimir@gmail.com> wrote:
> Philippe>GUI mode
>
> Please pardon my ignorance, however can you clarify what is the main
> difference between GUI and non-GUI modes that causes GUI to fail?
>
I am not sure you ignore anything about that Vladimir :-)
Can you clarify your question if it's a real one.
> Well, I saw your comment at
> https://www.itcentralstation.com/product_reviews/apache-jmet
> er-review-33516-by-rahulsharma
> ,
> and I'm all ears to hear what the reviewer would reply.
>
> Is it just "non-GUI skips things like View Results Tree"?
>
This one is a big one. Did you see the last thread "High CPU utilization in
JMeter 3.x with HttpClient 4 leads to freeze" ?
Although user has checked "only errors" , he ends up facing issues due to
connect failure and possibly "bandwidth overload".
Once View Result Tree stacks Sample Results, we're done and JMeter will
freeze.
> If so, I don't see much sense in having different limits on the number of
> threads in GUI and non-GUI modes.
>
I think there should also be a limit on number of thread in GUI mode,
because over a certain number of thread you'll face GUI freeze even with
light Listeners like Summary Report.
And I don't see any reason in Debug for using GUI mode for more than 10
Threads.
>
> Some kind of "performance mode" (e.g. enabled by default) or "debug mode"
> (disabled by default) might help.
> For instance, in those modes JMeter would skip things like "View Results
> Tree" and other complex UI stuff.
>
> Does that make sense?
>
Yes, I find it a good idea but I am afraid of false bug reports.
Just see what happens with the Stripping mode enabled in 2.9. I still see
reports of people not understanding why in Distributed mode they don't see
the response.
I think we need to provide a OOTB solution (nothing to configure) that for
GUI works correclty for "debug mode" and blocks Load test mode.
But we can also add your mode which can suit some use cases.
>
> Vladimir
>
--
Cordialement.
Philippe Mouawad.
Re: GUI Mode : OOM Protection for View Results Tree or View Results
in Table
Posted by Vladimir Sitnikov <si...@gmail.com>.
Philippe>GUI mode
Please pardon my ignorance, however can you clarify what is the main
difference between GUI and non-GUI modes that causes GUI to fail?
Well, I saw your comment at
https://www.itcentralstation.com/product_reviews/apache-jmeter-review-33516-by-rahulsharma
,
and I'm all ears to hear what the reviewer would reply.
Is it just "non-GUI skips things like View Results Tree"?
If so, I don't see much sense in having different limits on the number of
threads in GUI and non-GUI modes.
Some kind of "performance mode" (e.g. enabled by default) or "debug mode"
(disabled by default) might help.
For instance, in those modes JMeter would skip things like "View Results
Tree" and other complex UI stuff.
Does that make sense?
Vladimir