You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomee.apache.org by Jens Zurawski <jz...@diegurus.de> on 2023/10/26 10:46:50 UTC
Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Hi altogether,
I'm new to this list and hope it's the right place to ask this question.
If not, and someone know the right place to ask this, please give advise.
I'm developing a big JSF Application for a customer. It's a long running
project and development and when I started with it, TomEE 8.0.6 was the
most recent Version of TomEE. The switch to Jakarta is planned for next
year, so I'm still on the 8.x path for the time being and wasn't able to
test if this problem still exists in the 9.x branch.
The problem:
On the customers site I'm with the latest TomEE version 8.0.15 (running
on Java 11) and everything works fine, because it is running in
production mode. But in my development environment I'm still stuck with
8.0.6, because there I need the development mode
(javax.faces.PROJECT_STAGE: Development). I need to be able to see
changes in facelets without restarting everything everytime, and to get
more detailed error messages.
My attempts to update my dev TomEE to something newer than 8.0.6 all
failed, because all (at least all I've tested so far) newer versions are
so incredibly slow in dev mode, that it's unbearable to use the
application. Several very looong seconds on every request (even the
little AJAX requests in a view) is simply not a practical environment.
After switching to Production mode everything works very fast, and all
my views have response times of very few ms (even the big ones with max.
around 400ms). When switching back to Development mode I have response
times of up to 30s on big views. Not with 8.0.6, there even in
development mode it's reasonable fast in not getting higher than 1s.
My question:
What causes this enormous performance degradation? I'm hoping, it is
just a configuration which now has another default value or the like. If
yes, maybe someone can point me in the right direction of where to find
this configuration? If it's not a configuration thing: what can I do to
get around this?
I haven't tested all Versions of TomEE from 8.0.6 to 8.0.15, so I can't
say at what version exactly this behaviour changes. If it helps or is
needed, I can make some tests to find the exact version where this
happens. Versions I've already tested are: 8.0.12, 8.0.13 and 8.0.15.
All of them are very slow in dev mode. If you need more information,
I'll try to provide it.
Thanks in advance for any help
cu
Jens
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by "Jonathan S. Fisher" <ex...@gmail.com>.
Just when I think I've started to learn everything about the JVM,
someone posts some strange esoteric JVM switch... I've never even seen
those before. More details about what they do here:
https://bugs.openjdk.org/browse/JDK-8301001
I also don't know jack about crap about Windows, but if someone said
they were having this problem on Linux/OSX, I would say they need to
monitor class compilations and file descriptor limits. VisualVM has
the ability to monitor class compilations, I would take a peek at that
too.
On Fri, Oct 27, 2023 at 10:12 AM Richard Zowalla <ri...@zowalla.com> wrote:
>
> Great news! (though I am not a Windows user anymore).
>
> You should be able to build TomEE on Windows, too. If that does not
> work out-of-the-box, you could try to build within a docker container
> or via git-bash in Windows
>
> (I am not 100% sure a Windows build will work out of the box)
>
> I agree, that it might complicate things for testing purpose as it
> increases the amount of cycles needed, but that are already very good
> insights!
>
> Gruß
> Richard
>
>
> Am Freitag, dem 27.10.2023 um 16:55 +0200 schrieb Jens Zurawski:
> > I have a first result on this issue:
> >
> > The problem does not occur if TomEE is running on Linux. I've tested
> > my
> > own built 8.0.8, a freshly extracted 8.0.8 and 8.0.15 from the
> > releases,
> > and none of them showed the dev mode performance problem.
> >
> > Using TomEE 8.0.8 on Windows does show the dev mode performance
> > problem.
> > All of the tested ones, means the released 8.0.8 and my own built
> > 8.0.8
> > (the one I've built on Linux).
> >
> > My development environment is Windows 10 (Netbeans and therefore also
> > the TomEE for easy deployment and debugging). The problem does occur
> > on
> > stand alone started TomEE and also if started out of Netbeans, so
> > Netbeans can be sorted out as the root of the problem. Java Versions
> > 11,
> > 13, 15 and 17 tested, all the same, so Java should also not be the
> > problem. BTW: It is very unlikely a problem of my individual Windows
> > installation, because this problem also shows up on the Windows
> > system
> > of my colleague, which btw. was the root cause why I'm started to dig
> > deeper into this problem now and why I subscribed to this mailing
> > list ;-)
> >
> > I'm afraid, this will complicate my git bisect session a little bit
> > :-(
> >
> > However, I'll keep you posted.
> >
> > cu
> > Jens
> >
> >
> > Am 27.10.2023 um 13:12 schrieb Jens Zurawski:
> > > Thanks Richard (and Vicente),
> > >
> > > I've changed to JDK 8 and now the builds are successful (both 8.0.6
> > > and 8.0.8).
> > >
> > > I'll script me some test scripts to ease the build, run and test
> > > for
> > > every bisect step.
> > > A build needs around 3 to 4 minutes. Running a test with my
> > > application will need some manual interaction and checks, I guess
> > > also
> > > something around 4 minutes. This sums up to roughly 8 minutes per
> > > step.
> > >
> > > I've started a bisect and it told me:
> > > git bisect good 20441eb
> > > git bisect bad 4c8a616
> > > Bisecting: 136 revisions left to test after this (roughly 7 steps)
> > >
> > > That means, if everything works fine, I should be able to identify
> > > the
> > > commit after about an hour? (That's cool.... why nobody has told me
> > > about git bisect before? ;-) )
> > >
> > > Anyway, I will have to find some time for this, maybe I need a
> > > couple
> > > of days to find some.
> > > I will let you know about the outcome.
> > >
> > > cu
> > > Jens
> > >
> > >
> > > Am 27.10.2023 um 11:50 schrieb Richard Zowalla:
> > > > Hi,
> > > >
> > > > you need Java/JDK 8 to build TomEE 8.
> > > >
> > > > The Maven command is
> > > >
> > > > mvn --fail-at-end clean install -pl tomee/apache-tomee -am -
> > > > Dfile.encoding=UTF-8 -DskipTests=true
> > > >
> > > > You will find the tar.gz/zip in ./tomee/apache-tomee/target
> > > >
> > > > Gruß
> > > > Richard
> > > >
> > > > P.S. 8.0.7 is here:
> > > > https://archive.apache.org/dist/tomee/tomee-8.0.7/
> > > >
> > > > Am Freitag, dem 27.10.2023 um 10:50 +0200 schrieb Jens Zurawski:
> > > > > Hi Richard,
> > > > >
> > > > > to identify the "bad" version was easy. I didn't find a 8.0.7
> > > > > release
> > > > > on
> > > > > the download page, so I started with 8.0.8. And this version
> > > > > already
> > > > > has
> > > > > the slow dev mode. So for my problem I can say:
> > > > > good: 8.0.6
> > > > > bad: 8.0.8
> > > > >
> > > > > Now, I think, I need a little help from you. I tried to build a
> > > > > TomEE
> > > > > out of git (I've cloned the github repository
> > > > > https://github.com/apache/tomee.git). I've tried these two
> > > > > commits:
> > > > > 20441eb (tag: tomee-8.0.6)
> > > > > 4c8a616 (tag: tomee-project-8.0.8)
> > > > >
> > > > > But the build fails every time on "TomEE :: Container :: Core"
> > > > > with:
> > > > >
> > > > > [INFO] BUILD FAILURE
> > > > > [INFO]
> > > > > ---------------------------------------------------------------
> > > > > ------
> > > > > ---
> > > > > [INFO] Total time: 01:19 min
> > > > > [INFO] Finished at: 2023-10-27T10:29:40+02:00
> > > > > [INFO]
> > > > > ---------------------------------------------------------------
> > > > > ------
> > > > > ---
> > > > > [ERROR] Failed to execute goal
> > > > > org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile
> > > > > (default-compile) on project openejb-core: Compilation failure:
> > > > > Compilation failure:
> > > > > [ERROR]
> > > > > /sandbox/tomee/container/openejb-
> > > > > core/src/main/java/org/apache/openejb/core/OrbFactory.java:[21,
> > > > > 21]
> > > > > package org.omg.CORBA does not exist
> > > > > [ERROR]
> > > > > /sandbox/tomee/container/openejb-
> > > > > core/src/main/java/org/apache/openejb/core/OrbFactory.java:[24,
> > > > > 12]
> > > > > cannot find symbol
> > > > > [ERROR] symbol: class ORB
> > > > > [ERROR] location: class org.apache.openejb.core.OrbFactory
> > > > >
> > > > > I've tried to build with:
> > > > > mvn clean install
> > > > >
> > > > > My build environment is a linux ubuntu 20.04.6 LTS VM with:
> > > > > git 2.25.1
> > > > > Maven 3.6.3
> > > > > java 14.0.2
> > > > >
> > > > > Is it some easy thing to fix, or do I have to switch some
> > > > > versions of
> > > > > my
> > > > > environment?
> > > > >
> > > > > cu
> > > > > Jens
> > > > >
> > > > > Am 26.10.2023 um 19:48 schrieb Richard Zowalla:
> > > > > > Hi,
> > > > > >
> > > > > > without knowing code or having a reproducer, it might be not
> > > > > > that
> > > > > > obvious to find a reason / answer for the "academic problem"
> > > > > > ;-)
> > > > > >
> > > > > > If you really want to boil it down, I recommend the following
> > > > > > approach:
> > > > > >
> > > > > > (1) From your description it seems, that 8.0.6 is the last
> > > > > > known
> > > > > > "good"
> > > > > > version.
> > > > > >
> > > > > > (2) You've investigated versions starting from 8.0.12
> > > > > > onwards,
> > > > > > which
> > > > > > seems to be "bad".
> > > > > >
> > > > > > Following the approach recommended by Jonathan:
> > > > > >
> > > > > > - Can you check if it happens in 8.0.7 (8.0.8, ...) until we
> > > > > > find
> > > > > > the
> > > > > > first "bad" released version?
> > > > > >
> > > > > > - If we have that, you could start a "git bisect" session
> > > > > > between
> > > > > > 8.0.6
> > > > > > and the first version, which is known to be bad (reducing the
> > > > > > amount of
> > > > > > needed bisect cycles dramatically), so we find a commit /
> > > > > > reason
> > > > > > (if it
> > > > > > is in TomEE).
> > > > > >
> > > > > > - It would involve some quick builds without executing tests
> > > > > > (~10-
> > > > > > 15min
> > > > > > each) for each bisect step and testing on your side.
> > > > > >
> > > > > > Don't know if your inner monk is willing to go that route,
> > > > > > but I am
> > > > > > happy to help with the related Maven commands for quick
> > > > > > building :)
> > > > > >
> > > > > > Gruß
> > > > > > Richard
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Am Donnerstag, dem 26.10.2023 um 18:00 +0200 schrieb Jens
> > > > > > Zurawski:
> > > > > > > Hi Vicente,
> > > > > > >
> > > > > > > thank you for the tip. This is a good workaround. When I
> > > > > > > switch
> > > > > > > to
> > > > > > > Production mode and insert this configuration, it reloads
> > > > > > > the
> > > > > > > view
> > > > > > > from
> > > > > > > the file if it has changed. This means, I can use a decent
> > > > > > > version of
> > > > > > > TomEE in my dev environment.
> > > > > > >
> > > > > > > However, it doesn't completely solve my problem, as the
> > > > > > > error
> > > > > > > reporting
> > > > > > > isn't as verbose as in Dev mode as far as I can see. But I
> > > > > > > have
> > > > > > > to
> > > > > > > check, maybe it's enough for my development.
> > > > > > >
> > > > > > > And, well.... the academic problem: "Why is it still broken
> > > > > > > in
> > > > > > > Dev
> > > > > > > mode?" still runs circles through my brain ;-) Because if
> > > > > > > it's a
> > > > > > > problem
> > > > > > > of my coding, it might be a good idea to change this,
> > > > > > > before it
> > > > > > > might
> > > > > > > get a problem on Production mode, too, at some time in the
> > > > > > > future.
> > > > > > >
> > > > > > > But anyway, you've helped me a good step forward, thanks.
> > > > > > > cu
> > > > > > > Jens
> > > > > > >
> > > > > > >
> > > > > > > Am 26.10.2023 um 17:04 schrieb Vicente Rossello:
> > > > > > > > We are using the production stage with
> > > > > > > > javax.faces.FACELETS_REFRESH_PERIOD
> > > > > > > > = 1. It's faster and more similar to production, just try
> > > > > > > > it if
> > > > > > > > it
> > > > > > > > solves
> > > > > > > > your problem
> > > > > > > >
> > > > > > > > On Thu, Oct 26, 2023 at 4:18 PM Jens Zurawski
> > > > > > > > <jz...@diegurus.de>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > The one bundled with TomEE.
> > > > > > > > > myfaces 2.3.7 on TomEE 8.0.6
> > > > > > > > > and e.g. myfaces 2.3.9 on TomEE 8.0.12
> > > > > > > > >
> > > > > > > > > Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
> > > > > > > > > > Which JSF lib are you using?
> > > > > > > > > >
> > > > > > > > > > Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens
> > > > > > > > > > Zurawski
> > > > > > > > > > <jzu@diegurus.de
> > > > > > > > > > :
> > > > > > > > > > > Thank you for your quick reply, Jonathan.
> > > > > > > > > > >
> > > > > > > > > > > It is good to know, that normally the Dev mode
> > > > > > > > > > > should
> > > > > > > > > > > work as
> > > > > > > > > > > fast as
> > > > > > > > > before even with 8.0.15. So it obviously has something
> > > > > > > > > to do
> > > > > > > > > with
> > > > > > > > > my
> > > > > > > > > environment, configuration and/or code.
> > > > > > > > > > > And also thank you for the reference to tomitribe.
> > > > > > > > > > > Unfortunately I'm
> > > > > > > > > the only developer on this project, and it will be
> > > > > > > > > difficult
> > > > > > > > > to
> > > > > > > > > convince my
> > > > > > > > > customer to an additional support subscription, because
> > > > > > > > > he on
> > > > > > > > > production
> > > > > > > > > doesn't have issues ;-) But, ok, that's my problem, and
> > > > > > > > > maybe
> > > > > > > > > I'll consult
> > > > > > > > > tomitribe in the future for other projects.
> > > > > > > > > > > So, I'm afraid, for the near future I'm on my own
> > > > > > > > > > > to
> > > > > > > > > > > solve
> > > > > > > > > > > this
> > > > > > > > > problem. Therefore any hints to restrict the places
> > > > > > > > > where I
> > > > > > > > > should take a
> > > > > > > > > look will be more than welcome.
> > > > > > > > > > > I'm a senior programmer with 20 years of experience
> > > > > > > > > > > in
> > > > > > > > > > > Java
> > > > > > > > > > > programming
> > > > > > > > > (SE and JSP). But I'm relative new to
> > > > > > > > > JSF/JavaEE/JakartaEE
> > > > > > > > > and
> > > > > > > > > sometimes I
> > > > > > > > > have a hard time to understand which component of the
> > > > > > > > > whole
> > > > > > > > > stack
> > > > > > > > > is
> > > > > > > > > responsible for what.
> > > > > > > > > > > The project meanwhile has over a 1/4 million lines
> > > > > > > > > > > of
> > > > > > > > > > > code
> > > > > > > > > > > (most of
> > > > > > > > > them in the back-end, but also the front-end is big
> > > > > > > > > enough to
> > > > > > > > > not
> > > > > > > > > trace
> > > > > > > > > through every lane if I want to get to the point in
> > > > > > > > > this life
> > > > > > > > > :-D
> > > > > > > > > ).
> > > > > > > > > > > The application now runs for nearly 2 years on the
> > > > > > > > > > > customers
> > > > > > > > > > > site and
> > > > > > > > > is very reliable and fast in its daily usage. Apart
> > > > > > > > > from
> > > > > > > > > occasional updates
> > > > > > > > > every few month, it will never be restarted or need any
> > > > > > > > > manual
> > > > > > > > > maintenance
> > > > > > > > > at all. So I assume the code is not the badest on
> > > > > > > > > earth. At
> > > > > > > > > least
> > > > > > > > > because
> > > > > > > > > of that I'm able to out-rule things like memory leaks,
> > > > > > > > > gc
> > > > > > > > > issues
> > > > > > > > > or
> > > > > > > > > bottlenecks (max Heap is never reached, neither half of
> > > > > > > > > it,
> > > > > > > > > even
> > > > > > > > > after
> > > > > > > > > running for month).
> > > > > > > > > > > Of course there could be aspects of JSF which I may
> > > > > > > > > > > not
> > > > > > > > > > > have
> > > > > > > > > > > understood
> > > > > > > > > correctly until now. But because there are so many new
> > > > > > > > > things
> > > > > > > > > to
> > > > > > > > > a JSF
> > > > > > > > > beginner like me, I'm running out of ideas what this
> > > > > > > > > could
> > > > > > > > > be. It
> > > > > > > > > is
> > > > > > > > > something that worked well 'til TomEE 8.0.6 (Prod and
> > > > > > > > > Dev
> > > > > > > > > mode)
> > > > > > > > > and
> > > > > > > > > suddenly don't work anymore on 8.0.12 in Dev mode (but
> > > > > > > > > still
> > > > > > > > > works great in
> > > > > > > > > Prod mode). There are only minor versions in-between.
> > > > > > > > > So I
> > > > > > > > > think
> > > > > > > > > there were
> > > > > > > > > no revolutionary changes anywhere. My hope was (and
> > > > > > > > > still
> > > > > > > > > is),
> > > > > > > > > that anybody
> > > > > > > > > here on the list might have had a similar problem and
> > > > > > > > > has
> > > > > > > > > solved
> > > > > > > > > it, or
> > > > > > > > > maybe someone knows about some changes in Dev mode
> > > > > > > > > which
> > > > > > > > > could
> > > > > > > > > explain such
> > > > > > > > > behaviour (don't know, maybe xml parsing or component
> > > > > > > > > tree
> > > > > > > > > generation has
> > > > > > > > > changed, or scope handling of managed beans, or
> > > > > > > > > whatever
> > > > > > > > > could
> > > > > > > > > cause some
> > > > > > > > > code to be entered far more often than before, or some
> > > > > > > > > cache/pool
> > > > > > > > > was
> > > > > > > > > removed or has different behaviour... such things).
> > > > > > > > > Then it
> > > > > > > > > would
> > > > > > > > > be far
> > > > > > > > > more easy for me to track down the real problem.
> > > > > > > > > > > Anyway, when I get some spare time (currently have
> > > > > > > > > > > some
> > > > > > > > > > > deadlines in
> > > > > > > > > sight), I'll try to build a small test case. Either I
> > > > > > > > > can get
> > > > > > > > > a
> > > > > > > > > grip on the
> > > > > > > > > issue myself on doing this, or I can supply it to
> > > > > > > > > anyone who
> > > > > > > > > might want to
> > > > > > > > > have a look at it.
> > > > > > > > > > > cu
> > > > > > > > > > > Jens
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
> > > > > > > > > > > > > newer versions are so incredibly slow in dev
> > > > > > > > > > > > > mode
> > > > > > > > > > > > I can assure you that Dev/Prod mode works very
> > > > > > > > > > > > very
> > > > > > > > > > > > swiftly
> > > > > > > > > > > > on 8.0.15
> > > > > > > > > > > > for sizable applications. For example, I have a
> > > > > > > > > > > > giant
> > > > > > > > > > > > application
> > > > > > > > > > > > (thousands of LOC, 50+ jsf views) that has sub-
> > > > > > > > > > > > 8ms
> > > > > > > > > > > > response
> > > > > > > > > > > > times
> > > > > > > > > > > > (minus database time). More than likely they are
> > > > > > > > > > > > doing
> > > > > > > > > > > > something they
> > > > > > > > > > > > should not be doing with JSF apis and are causing
> > > > > > > > > > > > problems.
> > > > > > > > > > > >
> > > > > > > > > > > > I would check the obvious things first with
> > > > > > > > > > > > VisualVM:
> > > > > > > > > > > > heap
> > > > > > > > > > > > size, gc
> > > > > > > > > > > > frequency, etc. You can also run their profiler
> > > > > > > > > > > > and see
> > > > > > > > > > > > if
> > > > > > > > > > > > you can
> > > > > > > > > > > > spot anything immediately obvious.
> > > > > > > > > > > >
> > > > > > > > > > > > After that, the easiest way to locate the code
> > > > > > > > > > > > slowing
> > > > > > > > > > > > you
> > > > > > > > > > > > down is
> > > > > > > > > > > > bisection. Cut half the code out, run, and
> > > > > > > > > > > > continue
> > > > > > > > > > > > cutting
> > > > > > > > > > > > until you
> > > > > > > > > > > > can locate the problematic code. Then after that,
> > > > > > > > > > > > isolate a
> > > > > > > > > > > > test case
> > > > > > > > > > > > where flipping a boolean flag triggers the issue
> > > > > > > > > > > > and
> > > > > > > > > > > > post
> > > > > > > > > > > > the results
> > > > > > > > > > > > here.
> > > > > > > > > > > >
> > > > > > > > > > > > I would get ahold of:
> > > > > > > > > > > > https://www.tomitribe.com who are
> > > > > > > > > > > > literal
> > > > > > > > > > > > experts in this stuff. They offer professional
> > > > > > > > > > > > services
> > > > > > > > > > > > to
> > > > > > > > > > > > handle
> > > > > > > > > > > > exactly these situations and can work directly in
> > > > > > > > > > > > your
> > > > > > > > > > > > codebase to
> > > > > > > > > > > > help you find and fix the problem.
> > > > > > > > > > > >
> > > > > > > > > > > > On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski
> > > > > > > > > > > > <jz...@diegurus.de> wrote:
> > > > > > > > > > > > > Hi altogether,
> > > > > > > > > > > > >
> > > > > > > > > > > > > I'm new to this list and hope it's the right
> > > > > > > > > > > > > place to
> > > > > > > > > > > > > ask
> > > > > > > > > > > > > this
> > > > > > > > > question.
> > > > > > > > > > > > > If not, and someone know the right place to ask
> > > > > > > > > > > > > this,
> > > > > > > > > > > > > please give
> > > > > > > > > advise.
> > > > > > > > > > > > > I'm developing a big JSF Application for a
> > > > > > > > > > > > > customer.
> > > > > > > > > > > > > It's
> > > > > > > > > > > > > a long
> > > > > > > > > running
> > > > > > > > > > > > > project and development and when I started with
> > > > > > > > > > > > > it,
> > > > > > > > > > > > > TomEE
> > > > > > > > > > > > > 8.0.6 was
> > > > > > > > > the
> > > > > > > > > > > > > most recent Version of TomEE. The switch to
> > > > > > > > > > > > > Jakarta
> > > > > > > > > > > > > is
> > > > > > > > > > > > > planned for
> > > > > > > > > next
> > > > > > > > > > > > > year, so I'm still on the 8.x path for the time
> > > > > > > > > > > > > being
> > > > > > > > > > > > > and
> > > > > > > > > > > > > wasn't able
> > > > > > > > > to
> > > > > > > > > > > > > test if this problem still exists in the 9.x
> > > > > > > > > > > > > branch.
> > > > > > > > > > > > >
> > > > > > > > > > > > > The problem:
> > > > > > > > > > > > > On the customers site I'm with the latest TomEE
> > > > > > > > > > > > > version
> > > > > > > > > > > > > 8.0.15
> > > > > > > > > (running
> > > > > > > > > > > > > on Java 11) and everything works fine, because
> > > > > > > > > > > > > it is
> > > > > > > > > > > > > running in
> > > > > > > > > > > > > production mode. But in my development
> > > > > > > > > > > > > environment
> > > > > > > > > > > > > I'm
> > > > > > > > > > > > > still stuck
> > > > > > > > > with
> > > > > > > > > > > > > 8.0.6, because there I need the development
> > > > > > > > > > > > > mode
> > > > > > > > > > > > > (javax.faces.PROJECT_STAGE: Development). I
> > > > > > > > > > > > > need to
> > > > > > > > > > > > > be
> > > > > > > > > > > > > able to see
> > > > > > > > > > > > > changes in facelets without restarting
> > > > > > > > > > > > > everything
> > > > > > > > > > > > > everytime, and to
> > > > > > > > > get
> > > > > > > > > > > > > more detailed error messages.
> > > > > > > > > > > > > My attempts to update my dev TomEE to something
> > > > > > > > > > > > > newer
> > > > > > > > > > > > > than 8.0.6 all
> > > > > > > > > > > > > failed, because all (at least all I've tested
> > > > > > > > > > > > > so far)
> > > > > > > > > > > > > newer versions
> > > > > > > > > are
> > > > > > > > > > > > > so incredibly slow in dev mode, that it's
> > > > > > > > > > > > > unbearable
> > > > > > > > > > > > > to
> > > > > > > > > > > > > use the
> > > > > > > > > > > > > application. Several very looong seconds on
> > > > > > > > > > > > > every
> > > > > > > > > > > > > request
> > > > > > > > > > > > > (even the
> > > > > > > > > > > > > little AJAX requests in a view) is simply not a
> > > > > > > > > > > > > practical
> > > > > > > > > > > > > environment.
> > > > > > > > > > > > > After switching to Production mode everything
> > > > > > > > > > > > > works
> > > > > > > > > > > > > very
> > > > > > > > > > > > > fast, and all
> > > > > > > > > > > > > my views have response times of very few ms
> > > > > > > > > > > > > (even the
> > > > > > > > > > > > > big
> > > > > > > > > > > > > ones with
> > > > > > > > > max.
> > > > > > > > > > > > > around 400ms). When switching back to
> > > > > > > > > > > > > Development
> > > > > > > > > > > > > mode I
> > > > > > > > > > > > > have response
> > > > > > > > > > > > > times of up to 30s on big views. Not with
> > > > > > > > > > > > > 8.0.6,
> > > > > > > > > > > > > there
> > > > > > > > > > > > > even in
> > > > > > > > > > > > > development mode it's reasonable fast in not
> > > > > > > > > > > > > getting
> > > > > > > > > > > > > higher than 1s.
> > > > > > > > > > > > >
> > > > > > > > > > > > > My question:
> > > > > > > > > > > > > What causes this enormous performance
> > > > > > > > > > > > > degradation?
> > > > > > > > > > > > > I'm
> > > > > > > > > > > > > hoping, it is
> > > > > > > > > > > > > just a configuration which now has another
> > > > > > > > > > > > > default
> > > > > > > > > > > > > value
> > > > > > > > > > > > > or the like.
> > > > > > > > > If
> > > > > > > > > > > > > yes, maybe someone can point me in the right
> > > > > > > > > > > > > direction of
> > > > > > > > > > > > > where to
> > > > > > > > > find
> > > > > > > > > > > > > this configuration? If it's not a configuration
> > > > > > > > > > > > > thing:
> > > > > > > > > > > > > what can I do
> > > > > > > > > to
> > > > > > > > > > > > > get around this?
> > > > > > > > > > > > >
> > > > > > > > > > > > > I haven't tested all Versions of TomEE from
> > > > > > > > > > > > > 8.0.6 to
> > > > > > > > > > > > > 8.0.15, so I
> > > > > > > > > can't
> > > > > > > > > > > > > say at what version exactly this behaviour
> > > > > > > > > > > > > changes.
> > > > > > > > > > > > > If it
> > > > > > > > > > > > > helps or is
> > > > > > > > > > > > > needed, I can make some tests to find the exact
> > > > > > > > > > > > > version
> > > > > > > > > > > > > where this
> > > > > > > > > > > > > happens. Versions I've already tested are:
> > > > > > > > > > > > > 8.0.12,
> > > > > > > > > > > > > 8.0.13
> > > > > > > > > > > > > and 8.0.15.
> > > > > > > > > > > > > All of them are very slow in dev mode. If you
> > > > > > > > > > > > > need
> > > > > > > > > > > > > more
> > > > > > > > > > > > > information,
> > > > > > > > > > > > > I'll try to provide it.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks in advance for any help
> > > > > > > > > > > > > cu
> > > > > > > > > > > > > Jens
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > >
> >
>
--
Jonathan | exabrial@gmail.com
Pessimists, see a jar as half empty. Optimists, in contrast, see it as
half full.
Engineers, of course, understand the glass is twice as big as it needs to be.
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Richard Zowalla <ri...@zowalla.com>.
Sounds like it should be worth reporting it ;-)
Am 28. Oktober 2023 08:51:20 MESZ schrieb Vicente Rossello <co...@gmail.com>:
>The tomcat regression is probably introduced here:
>https://github.com/apache/tomcat/commit/08431bc0b895aa80e78e993a006cabb73aaa6490
>
>As it only affects windows and when there are a lot of files (and those
>files are constantly refreshed) it's not probably reported by anyone.
>
>
>
>On Sat, Oct 28, 2023 at 8:24 AM Vicente Rossello <co...@gmail.com>
>wrote:
>
>> Sorry, that was not correct.
>>
>> The allowLinking is at the resources level. So, in the
>> META-INF/context.xml file:
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>> <Context allowCasualMultipartParsing="true" reloadable="false" containerSciFilter="org.apache.tomcat.websocket.server.WsSci|org.apache.jasper.servlet.JasperInitializer"
>> parallelAnnotationScanning="true"
>> clearReferencesThreadLocals="false" skipMemoryLeakChecksOnJvmShutdown="true">
>> <Manager pathname=""/>
>>
>> <Resources archiveIndexStrategy="bloom" allowLinking="true"/>
>> </Context>
>>
>>
>> BTW, the slowness is something between checking the canonicalPath too many times and jdk12 that removed those canonCache by default. See https://bugs.openjdk.org/browse/JDK-8258036
>>
>>
>> On Sat, Oct 28, 2023 at 8:07 AM Vicente Rossello <co...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> It is a tomcat problem. The real problem is in DirResourcesSet. It is
>>> calling a lot of times File.getCanonicalPath(). This method is very slow in
>>> windows.
>>>
>>> What I did is extending the DirResourcesSet to skip processing the
>>> getCanonicalPath method so many times, but I really didn't understand the
>>> problem.
>>>
>>> A simpler and better solution is to set allowLinking="true" in the
>>> context.xml file. So tomcat does not validate the canonicalPath every
>>> single time, which is just too much.
>>>
>>> Best regards,
>>> Vicente
>>>
>>> On Fri, Oct 27, 2023 at 5:51 PM Jens Zurawski <jz...@diegurus.de> wrote:
>>>
>>>> My client is the last Windows System in my little universe. Due to some
>>>> constraints with my customers I have to deal with it for the time being.
>>>>
>>>> The switches, Vicente has provided, solved the problem for me so far.
>>>> But my "inner monk" is still not satisfied, because practical no one
>>>> knows of these switches, and after all, TomEE should do a good job on
>>>> Windows, too. And because 8.0.6 does a good job, and 8.0.8 breaks it to
>>>> some degree, we should at least try to figure out, what exactly causes
>>>> this break. Maybe there is an easy way to fix it, then it should be
>>>> fixed IMHO.
>>>>
>>>> So, I will go through the bisect session anyway, as it seems the best
>>>> possibility to get a grip on it. But I don't want to make these builds
>>>> on Windows. I'll build them on Linux and will try to ease my life a bit
>>>> with some scripting.
>>>>
>>>> cu
>>>> Jens
>>>>
>>>>
>>>> Am 27.10.2023 um 17:11 schrieb Richard Zowalla:
>>>> > Great news! (though I am not a Windows user anymore).
>>>> >
>>>> > You should be able to build TomEE on Windows, too. If that does not
>>>> > work out-of-the-box, you could try to build within a docker container
>>>> > or via git-bash in Windows
>>>> >
>>>> > (I am not 100% sure a Windows build will work out of the box)
>>>> >
>>>> > I agree, that it might complicate things for testing purpose as it
>>>> > increases the amount of cycles needed, but that are already very good
>>>> > insights!
>>>> >
>>>> > Gruß
>>>> > Richard
>>>> >
>>>> >
>>>> > Am Freitag, dem 27.10.2023 um 16:55 +0200 schrieb Jens Zurawski:
>>>> >> I have a first result on this issue:
>>>> >>
>>>> >> The problem does not occur if TomEE is running on Linux. I've tested
>>>> >> my
>>>> >> own built 8.0.8, a freshly extracted 8.0.8 and 8.0.15 from the
>>>> >> releases,
>>>> >> and none of them showed the dev mode performance problem.
>>>> >>
>>>> >> Using TomEE 8.0.8 on Windows does show the dev mode performance
>>>> >> problem.
>>>> >> All of the tested ones, means the released 8.0.8 and my own built
>>>> >> 8.0.8
>>>> >> (the one I've built on Linux).
>>>> >>
>>>> >> My development environment is Windows 10 (Netbeans and therefore also
>>>> >> the TomEE for easy deployment and debugging). The problem does occur
>>>> >> on
>>>> >> stand alone started TomEE and also if started out of Netbeans, so
>>>> >> Netbeans can be sorted out as the root of the problem. Java Versions
>>>> >> 11,
>>>> >> 13, 15 and 17 tested, all the same, so Java should also not be the
>>>> >> problem. BTW: It is very unlikely a problem of my individual Windows
>>>> >> installation, because this problem also shows up on the Windows
>>>> >> system
>>>> >> of my colleague, which btw. was the root cause why I'm started to dig
>>>> >> deeper into this problem now and why I subscribed to this mailing
>>>> >> list ;-)
>>>> >>
>>>> >> I'm afraid, this will complicate my git bisect session a little bit
>>>> >> :-(
>>>> >>
>>>> >> However, I'll keep you posted.
>>>> >>
>>>> >> cu
>>>> >> Jens
>>>> >>
>>>> >>
>>>> >> Am 27.10.2023 um 13:12 schrieb Jens Zurawski:
>>>> >>> Thanks Richard (and Vicente),
>>>> >>>
>>>> >>> I've changed to JDK 8 and now the builds are successful (both 8.0.6
>>>> >>> and 8.0.8).
>>>> >>>
>>>> >>> I'll script me some test scripts to ease the build, run and test
>>>> >>> for
>>>> >>> every bisect step.
>>>> >>> A build needs around 3 to 4 minutes. Running a test with my
>>>> >>> application will need some manual interaction and checks, I guess
>>>> >>> also
>>>> >>> something around 4 minutes. This sums up to roughly 8 minutes per
>>>> >>> step.
>>>> >>>
>>>> >>> I've started a bisect and it told me:
>>>> >>> git bisect good 20441eb
>>>> >>> git bisect bad 4c8a616
>>>> >>> Bisecting: 136 revisions left to test after this (roughly 7 steps)
>>>> >>>
>>>> >>> That means, if everything works fine, I should be able to identify
>>>> >>> the
>>>> >>> commit after about an hour? (That's cool.... why nobody has told me
>>>> >>> about git bisect before? ;-) )
>>>> >>>
>>>> >>> Anyway, I will have to find some time for this, maybe I need a
>>>> >>> couple
>>>> >>> of days to find some.
>>>> >>> I will let you know about the outcome.
>>>> >>>
>>>> >>> cu
>>>> >>> Jens
>>>> >>>
>>>> >>>
>>>> >>> Am 27.10.2023 um 11:50 schrieb Richard Zowalla:
>>>> >>>> Hi,
>>>> >>>>
>>>> >>>> you need Java/JDK 8 to build TomEE 8.
>>>> >>>>
>>>> >>>> The Maven command is
>>>> >>>>
>>>> >>>> mvn --fail-at-end clean install -pl tomee/apache-tomee -am -
>>>> >>>> Dfile.encoding=UTF-8 -DskipTests=true
>>>> >>>>
>>>> >>>> You will find the tar.gz/zip in ./tomee/apache-tomee/target
>>>> >>>>
>>>> >>>> Gruß
>>>> >>>> Richard
>>>> >>>>
>>>> >>>> P.S. 8.0.7 is here:
>>>> >>>> https://archive.apache.org/dist/tomee/tomee-8.0.7/
>>>> >>>>
>>>> >>>> Am Freitag, dem 27.10.2023 um 10:50 +0200 schrieb Jens Zurawski:
>>>> >>>>> Hi Richard,
>>>> >>>>>
>>>> >>>>> to identify the "bad" version was easy. I didn't find a 8.0.7
>>>> >>>>> release
>>>> >>>>> on
>>>> >>>>> the download page, so I started with 8.0.8. And this version
>>>> >>>>> already
>>>> >>>>> has
>>>> >>>>> the slow dev mode. So for my problem I can say:
>>>> >>>>> good: 8.0.6
>>>> >>>>> bad: 8.0.8
>>>> >>>>>
>>>> >>>>> Now, I think, I need a little help from you. I tried to build a
>>>> >>>>> TomEE
>>>> >>>>> out of git (I've cloned the github repository
>>>> >>>>> https://github.com/apache/tomee.git). I've tried these two
>>>> >>>>> commits:
>>>> >>>>> 20441eb (tag: tomee-8.0.6)
>>>> >>>>> 4c8a616 (tag: tomee-project-8.0.8)
>>>> >>>>>
>>>> >>>>> But the build fails every time on "TomEE :: Container :: Core"
>>>> >>>>> with:
>>>> >>>>>
>>>> >>>>> [INFO] BUILD FAILURE
>>>> >>>>> [INFO]
>>>> >>>>> ---------------------------------------------------------------
>>>> >>>>> ------
>>>> >>>>> ---
>>>> >>>>> [INFO] Total time: 01:19 min
>>>> >>>>> [INFO] Finished at: 2023-10-27T10:29:40+02:00
>>>> >>>>> [INFO]
>>>> >>>>> ---------------------------------------------------------------
>>>> >>>>> ------
>>>> >>>>> ---
>>>> >>>>> [ERROR] Failed to execute goal
>>>> >>>>> org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile
>>>> >>>>> (default-compile) on project openejb-core: Compilation failure:
>>>> >>>>> Compilation failure:
>>>> >>>>> [ERROR]
>>>> >>>>> /sandbox/tomee/container/openejb-
>>>> >>>>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[21,
>>>> >>>>> 21]
>>>> >>>>> package org.omg.CORBA does not exist
>>>> >>>>> [ERROR]
>>>> >>>>> /sandbox/tomee/container/openejb-
>>>> >>>>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[24,
>>>> >>>>> 12]
>>>> >>>>> cannot find symbol
>>>> >>>>> [ERROR] symbol: class ORB
>>>> >>>>> [ERROR] location: class org.apache.openejb.core.OrbFactory
>>>> >>>>>
>>>> >>>>> I've tried to build with:
>>>> >>>>> mvn clean install
>>>> >>>>>
>>>> >>>>> My build environment is a linux ubuntu 20.04.6 LTS VM with:
>>>> >>>>> git 2.25.1
>>>> >>>>> Maven 3.6.3
>>>> >>>>> java 14.0.2
>>>> >>>>>
>>>> >>>>> Is it some easy thing to fix, or do I have to switch some
>>>> >>>>> versions of
>>>> >>>>> my
>>>> >>>>> environment?
>>>> >>>>>
>>>> >>>>> cu
>>>> >>>>> Jens
>>>> >>>>>
>>>> >>>>> Am 26.10.2023 um 19:48 schrieb Richard Zowalla:
>>>> >>>>>> Hi,
>>>> >>>>>>
>>>> >>>>>> without knowing code or having a reproducer, it might be not
>>>> >>>>>> that
>>>> >>>>>> obvious to find a reason / answer for the "academic problem"
>>>> >>>>>> ;-)
>>>> >>>>>>
>>>> >>>>>> If you really want to boil it down, I recommend the following
>>>> >>>>>> approach:
>>>> >>>>>>
>>>> >>>>>> (1) From your description it seems, that 8.0.6 is the last
>>>> >>>>>> known
>>>> >>>>>> "good"
>>>> >>>>>> version.
>>>> >>>>>>
>>>> >>>>>> (2) You've investigated versions starting from 8.0.12
>>>> >>>>>> onwards,
>>>> >>>>>> which
>>>> >>>>>> seems to be "bad".
>>>> >>>>>>
>>>> >>>>>> Following the approach recommended by Jonathan:
>>>> >>>>>>
>>>> >>>>>> - Can you check if it happens in 8.0.7 (8.0.8, ...) until we
>>>> >>>>>> find
>>>> >>>>>> the
>>>> >>>>>> first "bad" released version?
>>>> >>>>>>
>>>> >>>>>> - If we have that, you could start a "git bisect" session
>>>> >>>>>> between
>>>> >>>>>> 8.0.6
>>>> >>>>>> and the first version, which is known to be bad (reducing the
>>>> >>>>>> amount of
>>>> >>>>>> needed bisect cycles dramatically), so we find a commit /
>>>> >>>>>> reason
>>>> >>>>>> (if it
>>>> >>>>>> is in TomEE).
>>>> >>>>>>
>>>> >>>>>> - It would involve some quick builds without executing tests
>>>> >>>>>> (~10-
>>>> >>>>>> 15min
>>>> >>>>>> each) for each bisect step and testing on your side.
>>>> >>>>>>
>>>> >>>>>> Don't know if your inner monk is willing to go that route,
>>>> >>>>>> but I am
>>>> >>>>>> happy to help with the related Maven commands for quick
>>>> >>>>>> building :)
>>>> >>>>>>
>>>> >>>>>> Gruß
>>>> >>>>>> Richard
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> Am Donnerstag, dem 26.10.2023 um 18:00 +0200 schrieb Jens
>>>> >>>>>> Zurawski:
>>>> >>>>>>> Hi Vicente,
>>>> >>>>>>>
>>>> >>>>>>> thank you for the tip. This is a good workaround. When I
>>>> >>>>>>> switch
>>>> >>>>>>> to
>>>> >>>>>>> Production mode and insert this configuration, it reloads
>>>> >>>>>>> the
>>>> >>>>>>> view
>>>> >>>>>>> from
>>>> >>>>>>> the file if it has changed. This means, I can use a decent
>>>> >>>>>>> version of
>>>> >>>>>>> TomEE in my dev environment.
>>>> >>>>>>>
>>>> >>>>>>> However, it doesn't completely solve my problem, as the
>>>> >>>>>>> error
>>>> >>>>>>> reporting
>>>> >>>>>>> isn't as verbose as in Dev mode as far as I can see. But I
>>>> >>>>>>> have
>>>> >>>>>>> to
>>>> >>>>>>> check, maybe it's enough for my development.
>>>> >>>>>>>
>>>> >>>>>>> And, well.... the academic problem: "Why is it still broken
>>>> >>>>>>> in
>>>> >>>>>>> Dev
>>>> >>>>>>> mode?" still runs circles through my brain ;-) Because if
>>>> >>>>>>> it's a
>>>> >>>>>>> problem
>>>> >>>>>>> of my coding, it might be a good idea to change this,
>>>> >>>>>>> before it
>>>> >>>>>>> might
>>>> >>>>>>> get a problem on Production mode, too, at some time in the
>>>> >>>>>>> future.
>>>> >>>>>>>
>>>> >>>>>>> But anyway, you've helped me a good step forward, thanks.
>>>> >>>>>>> cu
>>>> >>>>>>> Jens
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>> Am 26.10.2023 um 17:04 schrieb Vicente Rossello:
>>>> >>>>>>>> We are using the production stage with
>>>> >>>>>>>> javax.faces.FACELETS_REFRESH_PERIOD
>>>> >>>>>>>> = 1. It's faster and more similar to production, just try
>>>> >>>>>>>> it if
>>>> >>>>>>>> it
>>>> >>>>>>>> solves
>>>> >>>>>>>> your problem
>>>> >>>>>>>>
>>>> >>>>>>>> On Thu, Oct 26, 2023 at 4:18 PM Jens Zurawski
>>>> >>>>>>>> <jz...@diegurus.de>
>>>> >>>>>>>> wrote:
>>>> >>>>>>>>
>>>> >>>>>>>>> The one bundled with TomEE.
>>>> >>>>>>>>> myfaces 2.3.7 on TomEE 8.0.6
>>>> >>>>>>>>> and e.g. myfaces 2.3.9 on TomEE 8.0.12
>>>> >>>>>>>>>
>>>> >>>>>>>>> Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
>>>> >>>>>>>>>> Which JSF lib are you using?
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens
>>>> >>>>>>>>>> Zurawski
>>>> >>>>>>>>>> <jzu@diegurus.de
>>>> >>>>>>>>>> :
>>>> >>>>>>>>>>> Thank you for your quick reply, Jonathan.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> It is good to know, that normally the Dev mode
>>>> >>>>>>>>>>> should
>>>> >>>>>>>>>>> work as
>>>> >>>>>>>>>>> fast as
>>>> >>>>>>>>> before even with 8.0.15. So it obviously has something
>>>> >>>>>>>>> to do
>>>> >>>>>>>>> with
>>>> >>>>>>>>> my
>>>> >>>>>>>>> environment, configuration and/or code.
>>>> >>>>>>>>>>> And also thank you for the reference to tomitribe.
>>>> >>>>>>>>>>> Unfortunately I'm
>>>> >>>>>>>>> the only developer on this project, and it will be
>>>> >>>>>>>>> difficult
>>>> >>>>>>>>> to
>>>> >>>>>>>>> convince my
>>>> >>>>>>>>> customer to an additional support subscription, because
>>>> >>>>>>>>> he on
>>>> >>>>>>>>> production
>>>> >>>>>>>>> doesn't have issues ;-) But, ok, that's my problem, and
>>>> >>>>>>>>> maybe
>>>> >>>>>>>>> I'll consult
>>>> >>>>>>>>> tomitribe in the future for other projects.
>>>> >>>>>>>>>>> So, I'm afraid, for the near future I'm on my own
>>>> >>>>>>>>>>> to
>>>> >>>>>>>>>>> solve
>>>> >>>>>>>>>>> this
>>>> >>>>>>>>> problem. Therefore any hints to restrict the places
>>>> >>>>>>>>> where I
>>>> >>>>>>>>> should take a
>>>> >>>>>>>>> look will be more than welcome.
>>>> >>>>>>>>>>> I'm a senior programmer with 20 years of experience
>>>> >>>>>>>>>>> in
>>>> >>>>>>>>>>> Java
>>>> >>>>>>>>>>> programming
>>>> >>>>>>>>> (SE and JSP). But I'm relative new to
>>>> >>>>>>>>> JSF/JavaEE/JakartaEE
>>>> >>>>>>>>> and
>>>> >>>>>>>>> sometimes I
>>>> >>>>>>>>> have a hard time to understand which component of the
>>>> >>>>>>>>> whole
>>>> >>>>>>>>> stack
>>>> >>>>>>>>> is
>>>> >>>>>>>>> responsible for what.
>>>> >>>>>>>>>>> The project meanwhile has over a 1/4 million lines
>>>> >>>>>>>>>>> of
>>>> >>>>>>>>>>> code
>>>> >>>>>>>>>>> (most of
>>>> >>>>>>>>> them in the back-end, but also the front-end is big
>>>> >>>>>>>>> enough to
>>>> >>>>>>>>> not
>>>> >>>>>>>>> trace
>>>> >>>>>>>>> through every lane if I want to get to the point in
>>>> >>>>>>>>> this life
>>>> >>>>>>>>> :-D
>>>> >>>>>>>>> ).
>>>> >>>>>>>>>>> The application now runs for nearly 2 years on the
>>>> >>>>>>>>>>> customers
>>>> >>>>>>>>>>> site and
>>>> >>>>>>>>> is very reliable and fast in its daily usage. Apart
>>>> >>>>>>>>> from
>>>> >>>>>>>>> occasional updates
>>>> >>>>>>>>> every few month, it will never be restarted or need any
>>>> >>>>>>>>> manual
>>>> >>>>>>>>> maintenance
>>>> >>>>>>>>> at all. So I assume the code is not the badest on
>>>> >>>>>>>>> earth. At
>>>> >>>>>>>>> least
>>>> >>>>>>>>> because
>>>> >>>>>>>>> of that I'm able to out-rule things like memory leaks,
>>>> >>>>>>>>> gc
>>>> >>>>>>>>> issues
>>>> >>>>>>>>> or
>>>> >>>>>>>>> bottlenecks (max Heap is never reached, neither half of
>>>> >>>>>>>>> it,
>>>> >>>>>>>>> even
>>>> >>>>>>>>> after
>>>> >>>>>>>>> running for month).
>>>> >>>>>>>>>>> Of course there could be aspects of JSF which I may
>>>> >>>>>>>>>>> not
>>>> >>>>>>>>>>> have
>>>> >>>>>>>>>>> understood
>>>> >>>>>>>>> correctly until now. But because there are so many new
>>>> >>>>>>>>> things
>>>> >>>>>>>>> to
>>>> >>>>>>>>> a JSF
>>>> >>>>>>>>> beginner like me, I'm running out of ideas what this
>>>> >>>>>>>>> could
>>>> >>>>>>>>> be. It
>>>> >>>>>>>>> is
>>>> >>>>>>>>> something that worked well 'til TomEE 8.0.6 (Prod and
>>>> >>>>>>>>> Dev
>>>> >>>>>>>>> mode)
>>>> >>>>>>>>> and
>>>> >>>>>>>>> suddenly don't work anymore on 8.0.12 in Dev mode (but
>>>> >>>>>>>>> still
>>>> >>>>>>>>> works great in
>>>> >>>>>>>>> Prod mode). There are only minor versions in-between.
>>>> >>>>>>>>> So I
>>>> >>>>>>>>> think
>>>> >>>>>>>>> there were
>>>> >>>>>>>>> no revolutionary changes anywhere. My hope was (and
>>>> >>>>>>>>> still
>>>> >>>>>>>>> is),
>>>> >>>>>>>>> that anybody
>>>> >>>>>>>>> here on the list might have had a similar problem and
>>>> >>>>>>>>> has
>>>> >>>>>>>>> solved
>>>> >>>>>>>>> it, or
>>>> >>>>>>>>> maybe someone knows about some changes in Dev mode
>>>> >>>>>>>>> which
>>>> >>>>>>>>> could
>>>> >>>>>>>>> explain such
>>>> >>>>>>>>> behaviour (don't know, maybe xml parsing or component
>>>> >>>>>>>>> tree
>>>> >>>>>>>>> generation has
>>>> >>>>>>>>> changed, or scope handling of managed beans, or
>>>> >>>>>>>>> whatever
>>>> >>>>>>>>> could
>>>> >>>>>>>>> cause some
>>>> >>>>>>>>> code to be entered far more often than before, or some
>>>> >>>>>>>>> cache/pool
>>>> >>>>>>>>> was
>>>> >>>>>>>>> removed or has different behaviour... such things).
>>>> >>>>>>>>> Then it
>>>> >>>>>>>>> would
>>>> >>>>>>>>> be far
>>>> >>>>>>>>> more easy for me to track down the real problem.
>>>> >>>>>>>>>>> Anyway, when I get some spare time (currently have
>>>> >>>>>>>>>>> some
>>>> >>>>>>>>>>> deadlines in
>>>> >>>>>>>>> sight), I'll try to build a small test case. Either I
>>>> >>>>>>>>> can get
>>>> >>>>>>>>> a
>>>> >>>>>>>>> grip on the
>>>> >>>>>>>>> issue myself on doing this, or I can supply it to
>>>> >>>>>>>>> anyone who
>>>> >>>>>>>>> might want to
>>>> >>>>>>>>> have a look at it.
>>>> >>>>>>>>>>> cu
>>>> >>>>>>>>>>> Jens
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
>>>> >>>>>>>>>>>>> newer versions are so incredibly slow in dev
>>>> >>>>>>>>>>>>> mode
>>>> >>>>>>>>>>>> I can assure you that Dev/Prod mode works very
>>>> >>>>>>>>>>>> very
>>>> >>>>>>>>>>>> swiftly
>>>> >>>>>>>>>>>> on 8.0.15
>>>> >>>>>>>>>>>> for sizable applications. For example, I have a
>>>> >>>>>>>>>>>> giant
>>>> >>>>>>>>>>>> application
>>>> >>>>>>>>>>>> (thousands of LOC, 50+ jsf views) that has sub-
>>>> >>>>>>>>>>>> 8ms
>>>> >>>>>>>>>>>> response
>>>> >>>>>>>>>>>> times
>>>> >>>>>>>>>>>> (minus database time). More than likely they are
>>>> >>>>>>>>>>>> doing
>>>> >>>>>>>>>>>> something they
>>>> >>>>>>>>>>>> should not be doing with JSF apis and are causing
>>>> >>>>>>>>>>>> problems.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> I would check the obvious things first with
>>>> >>>>>>>>>>>> VisualVM:
>>>> >>>>>>>>>>>> heap
>>>> >>>>>>>>>>>> size, gc
>>>> >>>>>>>>>>>> frequency, etc. You can also run their profiler
>>>> >>>>>>>>>>>> and see
>>>> >>>>>>>>>>>> if
>>>> >>>>>>>>>>>> you can
>>>> >>>>>>>>>>>> spot anything immediately obvious.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> After that, the easiest way to locate the code
>>>> >>>>>>>>>>>> slowing
>>>> >>>>>>>>>>>> you
>>>> >>>>>>>>>>>> down is
>>>> >>>>>>>>>>>> bisection. Cut half the code out, run, and
>>>> >>>>>>>>>>>> continue
>>>> >>>>>>>>>>>> cutting
>>>> >>>>>>>>>>>> until you
>>>> >>>>>>>>>>>> can locate the problematic code. Then after that,
>>>> >>>>>>>>>>>> isolate a
>>>> >>>>>>>>>>>> test case
>>>> >>>>>>>>>>>> where flipping a boolean flag triggers the issue
>>>> >>>>>>>>>>>> and
>>>> >>>>>>>>>>>> post
>>>> >>>>>>>>>>>> the results
>>>> >>>>>>>>>>>> here.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> I would get ahold of:
>>>> >>>>>>>>>>>> https://www.tomitribe.com who are
>>>> >>>>>>>>>>>> literal
>>>> >>>>>>>>>>>> experts in this stuff. They offer professional
>>>> >>>>>>>>>>>> services
>>>> >>>>>>>>>>>> to
>>>> >>>>>>>>>>>> handle
>>>> >>>>>>>>>>>> exactly these situations and can work directly in
>>>> >>>>>>>>>>>> your
>>>> >>>>>>>>>>>> codebase to
>>>> >>>>>>>>>>>> help you find and fix the problem.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski
>>>> >>>>>>>>>>>> <jz...@diegurus.de> wrote:
>>>> >>>>>>>>>>>>> Hi altogether,
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> I'm new to this list and hope it's the right
>>>> >>>>>>>>>>>>> place to
>>>> >>>>>>>>>>>>> ask
>>>> >>>>>>>>>>>>> this
>>>> >>>>>>>>> question.
>>>> >>>>>>>>>>>>> If not, and someone know the right place to ask
>>>> >>>>>>>>>>>>> this,
>>>> >>>>>>>>>>>>> please give
>>>> >>>>>>>>> advise.
>>>> >>>>>>>>>>>>> I'm developing a big JSF Application for a
>>>> >>>>>>>>>>>>> customer.
>>>> >>>>>>>>>>>>> It's
>>>> >>>>>>>>>>>>> a long
>>>> >>>>>>>>> running
>>>> >>>>>>>>>>>>> project and development and when I started with
>>>> >>>>>>>>>>>>> it,
>>>> >>>>>>>>>>>>> TomEE
>>>> >>>>>>>>>>>>> 8.0.6 was
>>>> >>>>>>>>> the
>>>> >>>>>>>>>>>>> most recent Version of TomEE. The switch to
>>>> >>>>>>>>>>>>> Jakarta
>>>> >>>>>>>>>>>>> is
>>>> >>>>>>>>>>>>> planned for
>>>> >>>>>>>>> next
>>>> >>>>>>>>>>>>> year, so I'm still on the 8.x path for the time
>>>> >>>>>>>>>>>>> being
>>>> >>>>>>>>>>>>> and
>>>> >>>>>>>>>>>>> wasn't able
>>>> >>>>>>>>> to
>>>> >>>>>>>>>>>>> test if this problem still exists in the 9.x
>>>> >>>>>>>>>>>>> branch.
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> The problem:
>>>> >>>>>>>>>>>>> On the customers site I'm with the latest TomEE
>>>> >>>>>>>>>>>>> version
>>>> >>>>>>>>>>>>> 8.0.15
>>>> >>>>>>>>> (running
>>>> >>>>>>>>>>>>> on Java 11) and everything works fine, because
>>>> >>>>>>>>>>>>> it is
>>>> >>>>>>>>>>>>> running in
>>>> >>>>>>>>>>>>> production mode. But in my development
>>>> >>>>>>>>>>>>> environment
>>>> >>>>>>>>>>>>> I'm
>>>> >>>>>>>>>>>>> still stuck
>>>> >>>>>>>>> with
>>>> >>>>>>>>>>>>> 8.0.6, because there I need the development
>>>> >>>>>>>>>>>>> mode
>>>> >>>>>>>>>>>>> (javax.faces.PROJECT_STAGE: Development). I
>>>> >>>>>>>>>>>>> need to
>>>> >>>>>>>>>>>>> be
>>>> >>>>>>>>>>>>> able to see
>>>> >>>>>>>>>>>>> changes in facelets without restarting
>>>> >>>>>>>>>>>>> everything
>>>> >>>>>>>>>>>>> everytime, and to
>>>> >>>>>>>>> get
>>>> >>>>>>>>>>>>> more detailed error messages.
>>>> >>>>>>>>>>>>> My attempts to update my dev TomEE to something
>>>> >>>>>>>>>>>>> newer
>>>> >>>>>>>>>>>>> than 8.0.6 all
>>>> >>>>>>>>>>>>> failed, because all (at least all I've tested
>>>> >>>>>>>>>>>>> so far)
>>>> >>>>>>>>>>>>> newer versions
>>>> >>>>>>>>> are
>>>> >>>>>>>>>>>>> so incredibly slow in dev mode, that it's
>>>> >>>>>>>>>>>>> unbearable
>>>> >>>>>>>>>>>>> to
>>>> >>>>>>>>>>>>> use the
>>>> >>>>>>>>>>>>> application. Several very looong seconds on
>>>> >>>>>>>>>>>>> every
>>>> >>>>>>>>>>>>> request
>>>> >>>>>>>>>>>>> (even the
>>>> >>>>>>>>>>>>> little AJAX requests in a view) is simply not a
>>>> >>>>>>>>>>>>> practical
>>>> >>>>>>>>>>>>> environment.
>>>> >>>>>>>>>>>>> After switching to Production mode everything
>>>> >>>>>>>>>>>>> works
>>>> >>>>>>>>>>>>> very
>>>> >>>>>>>>>>>>> fast, and all
>>>> >>>>>>>>>>>>> my views have response times of very few ms
>>>> >>>>>>>>>>>>> (even the
>>>> >>>>>>>>>>>>> big
>>>> >>>>>>>>>>>>> ones with
>>>> >>>>>>>>> max.
>>>> >>>>>>>>>>>>> around 400ms). When switching back to
>>>> >>>>>>>>>>>>> Development
>>>> >>>>>>>>>>>>> mode I
>>>> >>>>>>>>>>>>> have response
>>>> >>>>>>>>>>>>> times of up to 30s on big views. Not with
>>>> >>>>>>>>>>>>> 8.0.6,
>>>> >>>>>>>>>>>>> there
>>>> >>>>>>>>>>>>> even in
>>>> >>>>>>>>>>>>> development mode it's reasonable fast in not
>>>> >>>>>>>>>>>>> getting
>>>> >>>>>>>>>>>>> higher than 1s.
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> My question:
>>>> >>>>>>>>>>>>> What causes this enormous performance
>>>> >>>>>>>>>>>>> degradation?
>>>> >>>>>>>>>>>>> I'm
>>>> >>>>>>>>>>>>> hoping, it is
>>>> >>>>>>>>>>>>> just a configuration which now has another
>>>> >>>>>>>>>>>>> default
>>>> >>>>>>>>>>>>> value
>>>> >>>>>>>>>>>>> or the like.
>>>> >>>>>>>>> If
>>>> >>>>>>>>>>>>> yes, maybe someone can point me in the right
>>>> >>>>>>>>>>>>> direction of
>>>> >>>>>>>>>>>>> where to
>>>> >>>>>>>>> find
>>>> >>>>>>>>>>>>> this configuration? If it's not a configuration
>>>> >>>>>>>>>>>>> thing:
>>>> >>>>>>>>>>>>> what can I do
>>>> >>>>>>>>> to
>>>> >>>>>>>>>>>>> get around this?
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> I haven't tested all Versions of TomEE from
>>>> >>>>>>>>>>>>> 8.0.6 to
>>>> >>>>>>>>>>>>> 8.0.15, so I
>>>> >>>>>>>>> can't
>>>> >>>>>>>>>>>>> say at what version exactly this behaviour
>>>> >>>>>>>>>>>>> changes.
>>>> >>>>>>>>>>>>> If it
>>>> >>>>>>>>>>>>> helps or is
>>>> >>>>>>>>>>>>> needed, I can make some tests to find the exact
>>>> >>>>>>>>>>>>> version
>>>> >>>>>>>>>>>>> where this
>>>> >>>>>>>>>>>>> happens. Versions I've already tested are:
>>>> >>>>>>>>>>>>> 8.0.12,
>>>> >>>>>>>>>>>>> 8.0.13
>>>> >>>>>>>>>>>>> and 8.0.15.
>>>> >>>>>>>>>>>>> All of them are very slow in dev mode. If you
>>>> >>>>>>>>>>>>> need
>>>> >>>>>>>>>>>>> more
>>>> >>>>>>>>>>>>> information,
>>>> >>>>>>>>>>>>> I'll try to provide it.
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> Thanks in advance for any help
>>>> >>>>>>>>>>>>> cu
>>>> >>>>>>>>>>>>> Jens
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>>
>>>>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Vicente Rossello <co...@gmail.com>.
The tomcat regression is probably introduced here:
https://github.com/apache/tomcat/commit/08431bc0b895aa80e78e993a006cabb73aaa6490
As it only affects windows and when there are a lot of files (and those
files are constantly refreshed) it's not probably reported by anyone.
On Sat, Oct 28, 2023 at 8:24 AM Vicente Rossello <co...@gmail.com>
wrote:
> Sorry, that was not correct.
>
> The allowLinking is at the resources level. So, in the
> META-INF/context.xml file:
>
> <?xml version="1.0" encoding="UTF-8"?>
> <Context allowCasualMultipartParsing="true" reloadable="false" containerSciFilter="org.apache.tomcat.websocket.server.WsSci|org.apache.jasper.servlet.JasperInitializer"
> parallelAnnotationScanning="true"
> clearReferencesThreadLocals="false" skipMemoryLeakChecksOnJvmShutdown="true">
> <Manager pathname=""/>
>
> <Resources archiveIndexStrategy="bloom" allowLinking="true"/>
> </Context>
>
>
> BTW, the slowness is something between checking the canonicalPath too many times and jdk12 that removed those canonCache by default. See https://bugs.openjdk.org/browse/JDK-8258036
>
>
> On Sat, Oct 28, 2023 at 8:07 AM Vicente Rossello <co...@gmail.com>
> wrote:
>
>> Hi,
>>
>> It is a tomcat problem. The real problem is in DirResourcesSet. It is
>> calling a lot of times File.getCanonicalPath(). This method is very slow in
>> windows.
>>
>> What I did is extending the DirResourcesSet to skip processing the
>> getCanonicalPath method so many times, but I really didn't understand the
>> problem.
>>
>> A simpler and better solution is to set allowLinking="true" in the
>> context.xml file. So tomcat does not validate the canonicalPath every
>> single time, which is just too much.
>>
>> Best regards,
>> Vicente
>>
>> On Fri, Oct 27, 2023 at 5:51 PM Jens Zurawski <jz...@diegurus.de> wrote:
>>
>>> My client is the last Windows System in my little universe. Due to some
>>> constraints with my customers I have to deal with it for the time being.
>>>
>>> The switches, Vicente has provided, solved the problem for me so far.
>>> But my "inner monk" is still not satisfied, because practical no one
>>> knows of these switches, and after all, TomEE should do a good job on
>>> Windows, too. And because 8.0.6 does a good job, and 8.0.8 breaks it to
>>> some degree, we should at least try to figure out, what exactly causes
>>> this break. Maybe there is an easy way to fix it, then it should be
>>> fixed IMHO.
>>>
>>> So, I will go through the bisect session anyway, as it seems the best
>>> possibility to get a grip on it. But I don't want to make these builds
>>> on Windows. I'll build them on Linux and will try to ease my life a bit
>>> with some scripting.
>>>
>>> cu
>>> Jens
>>>
>>>
>>> Am 27.10.2023 um 17:11 schrieb Richard Zowalla:
>>> > Great news! (though I am not a Windows user anymore).
>>> >
>>> > You should be able to build TomEE on Windows, too. If that does not
>>> > work out-of-the-box, you could try to build within a docker container
>>> > or via git-bash in Windows
>>> >
>>> > (I am not 100% sure a Windows build will work out of the box)
>>> >
>>> > I agree, that it might complicate things for testing purpose as it
>>> > increases the amount of cycles needed, but that are already very good
>>> > insights!
>>> >
>>> > Gruß
>>> > Richard
>>> >
>>> >
>>> > Am Freitag, dem 27.10.2023 um 16:55 +0200 schrieb Jens Zurawski:
>>> >> I have a first result on this issue:
>>> >>
>>> >> The problem does not occur if TomEE is running on Linux. I've tested
>>> >> my
>>> >> own built 8.0.8, a freshly extracted 8.0.8 and 8.0.15 from the
>>> >> releases,
>>> >> and none of them showed the dev mode performance problem.
>>> >>
>>> >> Using TomEE 8.0.8 on Windows does show the dev mode performance
>>> >> problem.
>>> >> All of the tested ones, means the released 8.0.8 and my own built
>>> >> 8.0.8
>>> >> (the one I've built on Linux).
>>> >>
>>> >> My development environment is Windows 10 (Netbeans and therefore also
>>> >> the TomEE for easy deployment and debugging). The problem does occur
>>> >> on
>>> >> stand alone started TomEE and also if started out of Netbeans, so
>>> >> Netbeans can be sorted out as the root of the problem. Java Versions
>>> >> 11,
>>> >> 13, 15 and 17 tested, all the same, so Java should also not be the
>>> >> problem. BTW: It is very unlikely a problem of my individual Windows
>>> >> installation, because this problem also shows up on the Windows
>>> >> system
>>> >> of my colleague, which btw. was the root cause why I'm started to dig
>>> >> deeper into this problem now and why I subscribed to this mailing
>>> >> list ;-)
>>> >>
>>> >> I'm afraid, this will complicate my git bisect session a little bit
>>> >> :-(
>>> >>
>>> >> However, I'll keep you posted.
>>> >>
>>> >> cu
>>> >> Jens
>>> >>
>>> >>
>>> >> Am 27.10.2023 um 13:12 schrieb Jens Zurawski:
>>> >>> Thanks Richard (and Vicente),
>>> >>>
>>> >>> I've changed to JDK 8 and now the builds are successful (both 8.0.6
>>> >>> and 8.0.8).
>>> >>>
>>> >>> I'll script me some test scripts to ease the build, run and test
>>> >>> for
>>> >>> every bisect step.
>>> >>> A build needs around 3 to 4 minutes. Running a test with my
>>> >>> application will need some manual interaction and checks, I guess
>>> >>> also
>>> >>> something around 4 minutes. This sums up to roughly 8 minutes per
>>> >>> step.
>>> >>>
>>> >>> I've started a bisect and it told me:
>>> >>> git bisect good 20441eb
>>> >>> git bisect bad 4c8a616
>>> >>> Bisecting: 136 revisions left to test after this (roughly 7 steps)
>>> >>>
>>> >>> That means, if everything works fine, I should be able to identify
>>> >>> the
>>> >>> commit after about an hour? (That's cool.... why nobody has told me
>>> >>> about git bisect before? ;-) )
>>> >>>
>>> >>> Anyway, I will have to find some time for this, maybe I need a
>>> >>> couple
>>> >>> of days to find some.
>>> >>> I will let you know about the outcome.
>>> >>>
>>> >>> cu
>>> >>> Jens
>>> >>>
>>> >>>
>>> >>> Am 27.10.2023 um 11:50 schrieb Richard Zowalla:
>>> >>>> Hi,
>>> >>>>
>>> >>>> you need Java/JDK 8 to build TomEE 8.
>>> >>>>
>>> >>>> The Maven command is
>>> >>>>
>>> >>>> mvn --fail-at-end clean install -pl tomee/apache-tomee -am -
>>> >>>> Dfile.encoding=UTF-8 -DskipTests=true
>>> >>>>
>>> >>>> You will find the tar.gz/zip in ./tomee/apache-tomee/target
>>> >>>>
>>> >>>> Gruß
>>> >>>> Richard
>>> >>>>
>>> >>>> P.S. 8.0.7 is here:
>>> >>>> https://archive.apache.org/dist/tomee/tomee-8.0.7/
>>> >>>>
>>> >>>> Am Freitag, dem 27.10.2023 um 10:50 +0200 schrieb Jens Zurawski:
>>> >>>>> Hi Richard,
>>> >>>>>
>>> >>>>> to identify the "bad" version was easy. I didn't find a 8.0.7
>>> >>>>> release
>>> >>>>> on
>>> >>>>> the download page, so I started with 8.0.8. And this version
>>> >>>>> already
>>> >>>>> has
>>> >>>>> the slow dev mode. So for my problem I can say:
>>> >>>>> good: 8.0.6
>>> >>>>> bad: 8.0.8
>>> >>>>>
>>> >>>>> Now, I think, I need a little help from you. I tried to build a
>>> >>>>> TomEE
>>> >>>>> out of git (I've cloned the github repository
>>> >>>>> https://github.com/apache/tomee.git). I've tried these two
>>> >>>>> commits:
>>> >>>>> 20441eb (tag: tomee-8.0.6)
>>> >>>>> 4c8a616 (tag: tomee-project-8.0.8)
>>> >>>>>
>>> >>>>> But the build fails every time on "TomEE :: Container :: Core"
>>> >>>>> with:
>>> >>>>>
>>> >>>>> [INFO] BUILD FAILURE
>>> >>>>> [INFO]
>>> >>>>> ---------------------------------------------------------------
>>> >>>>> ------
>>> >>>>> ---
>>> >>>>> [INFO] Total time: 01:19 min
>>> >>>>> [INFO] Finished at: 2023-10-27T10:29:40+02:00
>>> >>>>> [INFO]
>>> >>>>> ---------------------------------------------------------------
>>> >>>>> ------
>>> >>>>> ---
>>> >>>>> [ERROR] Failed to execute goal
>>> >>>>> org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile
>>> >>>>> (default-compile) on project openejb-core: Compilation failure:
>>> >>>>> Compilation failure:
>>> >>>>> [ERROR]
>>> >>>>> /sandbox/tomee/container/openejb-
>>> >>>>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[21,
>>> >>>>> 21]
>>> >>>>> package org.omg.CORBA does not exist
>>> >>>>> [ERROR]
>>> >>>>> /sandbox/tomee/container/openejb-
>>> >>>>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[24,
>>> >>>>> 12]
>>> >>>>> cannot find symbol
>>> >>>>> [ERROR] symbol: class ORB
>>> >>>>> [ERROR] location: class org.apache.openejb.core.OrbFactory
>>> >>>>>
>>> >>>>> I've tried to build with:
>>> >>>>> mvn clean install
>>> >>>>>
>>> >>>>> My build environment is a linux ubuntu 20.04.6 LTS VM with:
>>> >>>>> git 2.25.1
>>> >>>>> Maven 3.6.3
>>> >>>>> java 14.0.2
>>> >>>>>
>>> >>>>> Is it some easy thing to fix, or do I have to switch some
>>> >>>>> versions of
>>> >>>>> my
>>> >>>>> environment?
>>> >>>>>
>>> >>>>> cu
>>> >>>>> Jens
>>> >>>>>
>>> >>>>> Am 26.10.2023 um 19:48 schrieb Richard Zowalla:
>>> >>>>>> Hi,
>>> >>>>>>
>>> >>>>>> without knowing code or having a reproducer, it might be not
>>> >>>>>> that
>>> >>>>>> obvious to find a reason / answer for the "academic problem"
>>> >>>>>> ;-)
>>> >>>>>>
>>> >>>>>> If you really want to boil it down, I recommend the following
>>> >>>>>> approach:
>>> >>>>>>
>>> >>>>>> (1) From your description it seems, that 8.0.6 is the last
>>> >>>>>> known
>>> >>>>>> "good"
>>> >>>>>> version.
>>> >>>>>>
>>> >>>>>> (2) You've investigated versions starting from 8.0.12
>>> >>>>>> onwards,
>>> >>>>>> which
>>> >>>>>> seems to be "bad".
>>> >>>>>>
>>> >>>>>> Following the approach recommended by Jonathan:
>>> >>>>>>
>>> >>>>>> - Can you check if it happens in 8.0.7 (8.0.8, ...) until we
>>> >>>>>> find
>>> >>>>>> the
>>> >>>>>> first "bad" released version?
>>> >>>>>>
>>> >>>>>> - If we have that, you could start a "git bisect" session
>>> >>>>>> between
>>> >>>>>> 8.0.6
>>> >>>>>> and the first version, which is known to be bad (reducing the
>>> >>>>>> amount of
>>> >>>>>> needed bisect cycles dramatically), so we find a commit /
>>> >>>>>> reason
>>> >>>>>> (if it
>>> >>>>>> is in TomEE).
>>> >>>>>>
>>> >>>>>> - It would involve some quick builds without executing tests
>>> >>>>>> (~10-
>>> >>>>>> 15min
>>> >>>>>> each) for each bisect step and testing on your side.
>>> >>>>>>
>>> >>>>>> Don't know if your inner monk is willing to go that route,
>>> >>>>>> but I am
>>> >>>>>> happy to help with the related Maven commands for quick
>>> >>>>>> building :)
>>> >>>>>>
>>> >>>>>> Gruß
>>> >>>>>> Richard
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> Am Donnerstag, dem 26.10.2023 um 18:00 +0200 schrieb Jens
>>> >>>>>> Zurawski:
>>> >>>>>>> Hi Vicente,
>>> >>>>>>>
>>> >>>>>>> thank you for the tip. This is a good workaround. When I
>>> >>>>>>> switch
>>> >>>>>>> to
>>> >>>>>>> Production mode and insert this configuration, it reloads
>>> >>>>>>> the
>>> >>>>>>> view
>>> >>>>>>> from
>>> >>>>>>> the file if it has changed. This means, I can use a decent
>>> >>>>>>> version of
>>> >>>>>>> TomEE in my dev environment.
>>> >>>>>>>
>>> >>>>>>> However, it doesn't completely solve my problem, as the
>>> >>>>>>> error
>>> >>>>>>> reporting
>>> >>>>>>> isn't as verbose as in Dev mode as far as I can see. But I
>>> >>>>>>> have
>>> >>>>>>> to
>>> >>>>>>> check, maybe it's enough for my development.
>>> >>>>>>>
>>> >>>>>>> And, well.... the academic problem: "Why is it still broken
>>> >>>>>>> in
>>> >>>>>>> Dev
>>> >>>>>>> mode?" still runs circles through my brain ;-) Because if
>>> >>>>>>> it's a
>>> >>>>>>> problem
>>> >>>>>>> of my coding, it might be a good idea to change this,
>>> >>>>>>> before it
>>> >>>>>>> might
>>> >>>>>>> get a problem on Production mode, too, at some time in the
>>> >>>>>>> future.
>>> >>>>>>>
>>> >>>>>>> But anyway, you've helped me a good step forward, thanks.
>>> >>>>>>> cu
>>> >>>>>>> Jens
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> Am 26.10.2023 um 17:04 schrieb Vicente Rossello:
>>> >>>>>>>> We are using the production stage with
>>> >>>>>>>> javax.faces.FACELETS_REFRESH_PERIOD
>>> >>>>>>>> = 1. It's faster and more similar to production, just try
>>> >>>>>>>> it if
>>> >>>>>>>> it
>>> >>>>>>>> solves
>>> >>>>>>>> your problem
>>> >>>>>>>>
>>> >>>>>>>> On Thu, Oct 26, 2023 at 4:18 PM Jens Zurawski
>>> >>>>>>>> <jz...@diegurus.de>
>>> >>>>>>>> wrote:
>>> >>>>>>>>
>>> >>>>>>>>> The one bundled with TomEE.
>>> >>>>>>>>> myfaces 2.3.7 on TomEE 8.0.6
>>> >>>>>>>>> and e.g. myfaces 2.3.9 on TomEE 8.0.12
>>> >>>>>>>>>
>>> >>>>>>>>> Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
>>> >>>>>>>>>> Which JSF lib are you using?
>>> >>>>>>>>>>
>>> >>>>>>>>>> Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens
>>> >>>>>>>>>> Zurawski
>>> >>>>>>>>>> <jzu@diegurus.de
>>> >>>>>>>>>> :
>>> >>>>>>>>>>> Thank you for your quick reply, Jonathan.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> It is good to know, that normally the Dev mode
>>> >>>>>>>>>>> should
>>> >>>>>>>>>>> work as
>>> >>>>>>>>>>> fast as
>>> >>>>>>>>> before even with 8.0.15. So it obviously has something
>>> >>>>>>>>> to do
>>> >>>>>>>>> with
>>> >>>>>>>>> my
>>> >>>>>>>>> environment, configuration and/or code.
>>> >>>>>>>>>>> And also thank you for the reference to tomitribe.
>>> >>>>>>>>>>> Unfortunately I'm
>>> >>>>>>>>> the only developer on this project, and it will be
>>> >>>>>>>>> difficult
>>> >>>>>>>>> to
>>> >>>>>>>>> convince my
>>> >>>>>>>>> customer to an additional support subscription, because
>>> >>>>>>>>> he on
>>> >>>>>>>>> production
>>> >>>>>>>>> doesn't have issues ;-) But, ok, that's my problem, and
>>> >>>>>>>>> maybe
>>> >>>>>>>>> I'll consult
>>> >>>>>>>>> tomitribe in the future for other projects.
>>> >>>>>>>>>>> So, I'm afraid, for the near future I'm on my own
>>> >>>>>>>>>>> to
>>> >>>>>>>>>>> solve
>>> >>>>>>>>>>> this
>>> >>>>>>>>> problem. Therefore any hints to restrict the places
>>> >>>>>>>>> where I
>>> >>>>>>>>> should take a
>>> >>>>>>>>> look will be more than welcome.
>>> >>>>>>>>>>> I'm a senior programmer with 20 years of experience
>>> >>>>>>>>>>> in
>>> >>>>>>>>>>> Java
>>> >>>>>>>>>>> programming
>>> >>>>>>>>> (SE and JSP). But I'm relative new to
>>> >>>>>>>>> JSF/JavaEE/JakartaEE
>>> >>>>>>>>> and
>>> >>>>>>>>> sometimes I
>>> >>>>>>>>> have a hard time to understand which component of the
>>> >>>>>>>>> whole
>>> >>>>>>>>> stack
>>> >>>>>>>>> is
>>> >>>>>>>>> responsible for what.
>>> >>>>>>>>>>> The project meanwhile has over a 1/4 million lines
>>> >>>>>>>>>>> of
>>> >>>>>>>>>>> code
>>> >>>>>>>>>>> (most of
>>> >>>>>>>>> them in the back-end, but also the front-end is big
>>> >>>>>>>>> enough to
>>> >>>>>>>>> not
>>> >>>>>>>>> trace
>>> >>>>>>>>> through every lane if I want to get to the point in
>>> >>>>>>>>> this life
>>> >>>>>>>>> :-D
>>> >>>>>>>>> ).
>>> >>>>>>>>>>> The application now runs for nearly 2 years on the
>>> >>>>>>>>>>> customers
>>> >>>>>>>>>>> site and
>>> >>>>>>>>> is very reliable and fast in its daily usage. Apart
>>> >>>>>>>>> from
>>> >>>>>>>>> occasional updates
>>> >>>>>>>>> every few month, it will never be restarted or need any
>>> >>>>>>>>> manual
>>> >>>>>>>>> maintenance
>>> >>>>>>>>> at all. So I assume the code is not the badest on
>>> >>>>>>>>> earth. At
>>> >>>>>>>>> least
>>> >>>>>>>>> because
>>> >>>>>>>>> of that I'm able to out-rule things like memory leaks,
>>> >>>>>>>>> gc
>>> >>>>>>>>> issues
>>> >>>>>>>>> or
>>> >>>>>>>>> bottlenecks (max Heap is never reached, neither half of
>>> >>>>>>>>> it,
>>> >>>>>>>>> even
>>> >>>>>>>>> after
>>> >>>>>>>>> running for month).
>>> >>>>>>>>>>> Of course there could be aspects of JSF which I may
>>> >>>>>>>>>>> not
>>> >>>>>>>>>>> have
>>> >>>>>>>>>>> understood
>>> >>>>>>>>> correctly until now. But because there are so many new
>>> >>>>>>>>> things
>>> >>>>>>>>> to
>>> >>>>>>>>> a JSF
>>> >>>>>>>>> beginner like me, I'm running out of ideas what this
>>> >>>>>>>>> could
>>> >>>>>>>>> be. It
>>> >>>>>>>>> is
>>> >>>>>>>>> something that worked well 'til TomEE 8.0.6 (Prod and
>>> >>>>>>>>> Dev
>>> >>>>>>>>> mode)
>>> >>>>>>>>> and
>>> >>>>>>>>> suddenly don't work anymore on 8.0.12 in Dev mode (but
>>> >>>>>>>>> still
>>> >>>>>>>>> works great in
>>> >>>>>>>>> Prod mode). There are only minor versions in-between.
>>> >>>>>>>>> So I
>>> >>>>>>>>> think
>>> >>>>>>>>> there were
>>> >>>>>>>>> no revolutionary changes anywhere. My hope was (and
>>> >>>>>>>>> still
>>> >>>>>>>>> is),
>>> >>>>>>>>> that anybody
>>> >>>>>>>>> here on the list might have had a similar problem and
>>> >>>>>>>>> has
>>> >>>>>>>>> solved
>>> >>>>>>>>> it, or
>>> >>>>>>>>> maybe someone knows about some changes in Dev mode
>>> >>>>>>>>> which
>>> >>>>>>>>> could
>>> >>>>>>>>> explain such
>>> >>>>>>>>> behaviour (don't know, maybe xml parsing or component
>>> >>>>>>>>> tree
>>> >>>>>>>>> generation has
>>> >>>>>>>>> changed, or scope handling of managed beans, or
>>> >>>>>>>>> whatever
>>> >>>>>>>>> could
>>> >>>>>>>>> cause some
>>> >>>>>>>>> code to be entered far more often than before, or some
>>> >>>>>>>>> cache/pool
>>> >>>>>>>>> was
>>> >>>>>>>>> removed or has different behaviour... such things).
>>> >>>>>>>>> Then it
>>> >>>>>>>>> would
>>> >>>>>>>>> be far
>>> >>>>>>>>> more easy for me to track down the real problem.
>>> >>>>>>>>>>> Anyway, when I get some spare time (currently have
>>> >>>>>>>>>>> some
>>> >>>>>>>>>>> deadlines in
>>> >>>>>>>>> sight), I'll try to build a small test case. Either I
>>> >>>>>>>>> can get
>>> >>>>>>>>> a
>>> >>>>>>>>> grip on the
>>> >>>>>>>>> issue myself on doing this, or I can supply it to
>>> >>>>>>>>> anyone who
>>> >>>>>>>>> might want to
>>> >>>>>>>>> have a look at it.
>>> >>>>>>>>>>> cu
>>> >>>>>>>>>>> Jens
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
>>> >>>>>>>>>>>>> newer versions are so incredibly slow in dev
>>> >>>>>>>>>>>>> mode
>>> >>>>>>>>>>>> I can assure you that Dev/Prod mode works very
>>> >>>>>>>>>>>> very
>>> >>>>>>>>>>>> swiftly
>>> >>>>>>>>>>>> on 8.0.15
>>> >>>>>>>>>>>> for sizable applications. For example, I have a
>>> >>>>>>>>>>>> giant
>>> >>>>>>>>>>>> application
>>> >>>>>>>>>>>> (thousands of LOC, 50+ jsf views) that has sub-
>>> >>>>>>>>>>>> 8ms
>>> >>>>>>>>>>>> response
>>> >>>>>>>>>>>> times
>>> >>>>>>>>>>>> (minus database time). More than likely they are
>>> >>>>>>>>>>>> doing
>>> >>>>>>>>>>>> something they
>>> >>>>>>>>>>>> should not be doing with JSF apis and are causing
>>> >>>>>>>>>>>> problems.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> I would check the obvious things first with
>>> >>>>>>>>>>>> VisualVM:
>>> >>>>>>>>>>>> heap
>>> >>>>>>>>>>>> size, gc
>>> >>>>>>>>>>>> frequency, etc. You can also run their profiler
>>> >>>>>>>>>>>> and see
>>> >>>>>>>>>>>> if
>>> >>>>>>>>>>>> you can
>>> >>>>>>>>>>>> spot anything immediately obvious.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> After that, the easiest way to locate the code
>>> >>>>>>>>>>>> slowing
>>> >>>>>>>>>>>> you
>>> >>>>>>>>>>>> down is
>>> >>>>>>>>>>>> bisection. Cut half the code out, run, and
>>> >>>>>>>>>>>> continue
>>> >>>>>>>>>>>> cutting
>>> >>>>>>>>>>>> until you
>>> >>>>>>>>>>>> can locate the problematic code. Then after that,
>>> >>>>>>>>>>>> isolate a
>>> >>>>>>>>>>>> test case
>>> >>>>>>>>>>>> where flipping a boolean flag triggers the issue
>>> >>>>>>>>>>>> and
>>> >>>>>>>>>>>> post
>>> >>>>>>>>>>>> the results
>>> >>>>>>>>>>>> here.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> I would get ahold of:
>>> >>>>>>>>>>>> https://www.tomitribe.com who are
>>> >>>>>>>>>>>> literal
>>> >>>>>>>>>>>> experts in this stuff. They offer professional
>>> >>>>>>>>>>>> services
>>> >>>>>>>>>>>> to
>>> >>>>>>>>>>>> handle
>>> >>>>>>>>>>>> exactly these situations and can work directly in
>>> >>>>>>>>>>>> your
>>> >>>>>>>>>>>> codebase to
>>> >>>>>>>>>>>> help you find and fix the problem.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski
>>> >>>>>>>>>>>> <jz...@diegurus.de> wrote:
>>> >>>>>>>>>>>>> Hi altogether,
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> I'm new to this list and hope it's the right
>>> >>>>>>>>>>>>> place to
>>> >>>>>>>>>>>>> ask
>>> >>>>>>>>>>>>> this
>>> >>>>>>>>> question.
>>> >>>>>>>>>>>>> If not, and someone know the right place to ask
>>> >>>>>>>>>>>>> this,
>>> >>>>>>>>>>>>> please give
>>> >>>>>>>>> advise.
>>> >>>>>>>>>>>>> I'm developing a big JSF Application for a
>>> >>>>>>>>>>>>> customer.
>>> >>>>>>>>>>>>> It's
>>> >>>>>>>>>>>>> a long
>>> >>>>>>>>> running
>>> >>>>>>>>>>>>> project and development and when I started with
>>> >>>>>>>>>>>>> it,
>>> >>>>>>>>>>>>> TomEE
>>> >>>>>>>>>>>>> 8.0.6 was
>>> >>>>>>>>> the
>>> >>>>>>>>>>>>> most recent Version of TomEE. The switch to
>>> >>>>>>>>>>>>> Jakarta
>>> >>>>>>>>>>>>> is
>>> >>>>>>>>>>>>> planned for
>>> >>>>>>>>> next
>>> >>>>>>>>>>>>> year, so I'm still on the 8.x path for the time
>>> >>>>>>>>>>>>> being
>>> >>>>>>>>>>>>> and
>>> >>>>>>>>>>>>> wasn't able
>>> >>>>>>>>> to
>>> >>>>>>>>>>>>> test if this problem still exists in the 9.x
>>> >>>>>>>>>>>>> branch.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> The problem:
>>> >>>>>>>>>>>>> On the customers site I'm with the latest TomEE
>>> >>>>>>>>>>>>> version
>>> >>>>>>>>>>>>> 8.0.15
>>> >>>>>>>>> (running
>>> >>>>>>>>>>>>> on Java 11) and everything works fine, because
>>> >>>>>>>>>>>>> it is
>>> >>>>>>>>>>>>> running in
>>> >>>>>>>>>>>>> production mode. But in my development
>>> >>>>>>>>>>>>> environment
>>> >>>>>>>>>>>>> I'm
>>> >>>>>>>>>>>>> still stuck
>>> >>>>>>>>> with
>>> >>>>>>>>>>>>> 8.0.6, because there I need the development
>>> >>>>>>>>>>>>> mode
>>> >>>>>>>>>>>>> (javax.faces.PROJECT_STAGE: Development). I
>>> >>>>>>>>>>>>> need to
>>> >>>>>>>>>>>>> be
>>> >>>>>>>>>>>>> able to see
>>> >>>>>>>>>>>>> changes in facelets without restarting
>>> >>>>>>>>>>>>> everything
>>> >>>>>>>>>>>>> everytime, and to
>>> >>>>>>>>> get
>>> >>>>>>>>>>>>> more detailed error messages.
>>> >>>>>>>>>>>>> My attempts to update my dev TomEE to something
>>> >>>>>>>>>>>>> newer
>>> >>>>>>>>>>>>> than 8.0.6 all
>>> >>>>>>>>>>>>> failed, because all (at least all I've tested
>>> >>>>>>>>>>>>> so far)
>>> >>>>>>>>>>>>> newer versions
>>> >>>>>>>>> are
>>> >>>>>>>>>>>>> so incredibly slow in dev mode, that it's
>>> >>>>>>>>>>>>> unbearable
>>> >>>>>>>>>>>>> to
>>> >>>>>>>>>>>>> use the
>>> >>>>>>>>>>>>> application. Several very looong seconds on
>>> >>>>>>>>>>>>> every
>>> >>>>>>>>>>>>> request
>>> >>>>>>>>>>>>> (even the
>>> >>>>>>>>>>>>> little AJAX requests in a view) is simply not a
>>> >>>>>>>>>>>>> practical
>>> >>>>>>>>>>>>> environment.
>>> >>>>>>>>>>>>> After switching to Production mode everything
>>> >>>>>>>>>>>>> works
>>> >>>>>>>>>>>>> very
>>> >>>>>>>>>>>>> fast, and all
>>> >>>>>>>>>>>>> my views have response times of very few ms
>>> >>>>>>>>>>>>> (even the
>>> >>>>>>>>>>>>> big
>>> >>>>>>>>>>>>> ones with
>>> >>>>>>>>> max.
>>> >>>>>>>>>>>>> around 400ms). When switching back to
>>> >>>>>>>>>>>>> Development
>>> >>>>>>>>>>>>> mode I
>>> >>>>>>>>>>>>> have response
>>> >>>>>>>>>>>>> times of up to 30s on big views. Not with
>>> >>>>>>>>>>>>> 8.0.6,
>>> >>>>>>>>>>>>> there
>>> >>>>>>>>>>>>> even in
>>> >>>>>>>>>>>>> development mode it's reasonable fast in not
>>> >>>>>>>>>>>>> getting
>>> >>>>>>>>>>>>> higher than 1s.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> My question:
>>> >>>>>>>>>>>>> What causes this enormous performance
>>> >>>>>>>>>>>>> degradation?
>>> >>>>>>>>>>>>> I'm
>>> >>>>>>>>>>>>> hoping, it is
>>> >>>>>>>>>>>>> just a configuration which now has another
>>> >>>>>>>>>>>>> default
>>> >>>>>>>>>>>>> value
>>> >>>>>>>>>>>>> or the like.
>>> >>>>>>>>> If
>>> >>>>>>>>>>>>> yes, maybe someone can point me in the right
>>> >>>>>>>>>>>>> direction of
>>> >>>>>>>>>>>>> where to
>>> >>>>>>>>> find
>>> >>>>>>>>>>>>> this configuration? If it's not a configuration
>>> >>>>>>>>>>>>> thing:
>>> >>>>>>>>>>>>> what can I do
>>> >>>>>>>>> to
>>> >>>>>>>>>>>>> get around this?
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> I haven't tested all Versions of TomEE from
>>> >>>>>>>>>>>>> 8.0.6 to
>>> >>>>>>>>>>>>> 8.0.15, so I
>>> >>>>>>>>> can't
>>> >>>>>>>>>>>>> say at what version exactly this behaviour
>>> >>>>>>>>>>>>> changes.
>>> >>>>>>>>>>>>> If it
>>> >>>>>>>>>>>>> helps or is
>>> >>>>>>>>>>>>> needed, I can make some tests to find the exact
>>> >>>>>>>>>>>>> version
>>> >>>>>>>>>>>>> where this
>>> >>>>>>>>>>>>> happens. Versions I've already tested are:
>>> >>>>>>>>>>>>> 8.0.12,
>>> >>>>>>>>>>>>> 8.0.13
>>> >>>>>>>>>>>>> and 8.0.15.
>>> >>>>>>>>>>>>> All of them are very slow in dev mode. If you
>>> >>>>>>>>>>>>> need
>>> >>>>>>>>>>>>> more
>>> >>>>>>>>>>>>> information,
>>> >>>>>>>>>>>>> I'll try to provide it.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> Thanks in advance for any help
>>> >>>>>>>>>>>>> cu
>>> >>>>>>>>>>>>> Jens
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>>
>>>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Jens Zurawski <jz...@diegurus.de>.
Thank you very much, Vicente. I've only just added a comment with some
infos.
Let's wait and see how they will react to it.
cu
Jens
Am 02.11.2023 um 13:49 schrieb Vicente Rossello:
> https://bz.apache.org/bugzilla/show_bug.cgi?id=68054
>
> Let's see how it works
>
> On Tue, Oct 31, 2023 at 3:04 PM Vicente Rossello <co...@gmail.com>
> wrote:
>
>> I will try to open a bug. An obvious fix is to only make 1 call to
>> getCannonicalPath for the same file. That would halve the time.
>>
>> After that I guess is tomcat team decission.... maybe emitting a warning
>> that allowLinking=false in windows is very slow for jsf development ... or
>> they may have more ideas
>>
>> El mar, 31 oct 2023, 14:20, Jens Zurawski <jz...@diegurus.de> escribió:
>>
>>> Ok, after 4 hours of bisecting, checking out, building and testing (I've
>>> needed three bisect sessions, because the first one wasn't successful
>>> due to build failures of some revisions and I had to manually find some
>>> other revisions to bisect) I can definitely identify the commit which
>>> introduces the DevMode performance issue. It is:
>>>
>>> 0325d91 TOMEE-3773 - Update Tomcat to 9.0.50
>>>
>>> I've cross checked with the immediate commit before (277fc71) and it's
>>> clear:
>>> 277fc71 good
>>> 0325d91 bad
>>>
>>> So, indeed, tomcat is the root cause of the problem. But, I don't think
>>> it will be successful to create an issue in tomcat. A far as I can see
>>> in the diffs of tomcat 9.0.48 to 9.0.50, I assume, it already was a bug
>>> fix in /java/org/apache/catalina/webresources/DirResourceSet.java. In
>>> that class new code was inserted to skip symbolic links in
>>> ServletContext.getResourcePaths(), which, I think, is important as a
>>> security measure. Therefore, most likely they will not be willing to
>>> revert that (I wouldn't recommend that, either)
>>>
>>> Long story short: It's not a bug it's a feature ;-)
>>>
>>> I guess, JSF in DevMode is doing a lot more calls to getResourcePaths()
>>> hence suffering from this fix. Because ProdMode is not affected by this
>>> I can live with it. Development environments are special in any case,
>>> and to allow the dev tomcat following symlinks in that environment is
>>> ok, from my pov. Just enable the allowLinking="true" and it works.
>>>
>>> At least my "academic problem" is solved and my "inner monk" is
>>> satisfied and I don't think I will walk down this route any further.
>>>
>>> Many thanks to all of you who have helped me on this issue.
>>>
>>> cu
>>> Jens
>>>
>>> Am 28.10.2023 um 15:17 schrieb Jens Zurawski:
>>>> Yes, this may be one of the next steps here. But, I'm afraid, I have
>>>> not enough info for them right now.
>>>> Neither do I have a small test case to reproduce the problem, nor can
>>>> I point them exact enough to what the problem might be. Also, maybe
>>>> it's only coincidence, and the real problem is somewhere inside
>>>> myfaces. The fact that ProdMode is not affected at all (at least not
>>>> enough to measure any difference) and it only arises in JSF DevMode
>>>> could point in that direction.
>>>>
>>>> So, before I go and open an issue on Tomcat, I want to try to narrow
>>>> down it more. At least to the point where I can say: look, this commit
>>>> in TomEE introduced it. And even better: here is a small test case to
>>>> reproduce it.
>>>>
>>>> cu
>>>> Jens
>>>>
>>>>
>>>> Am 28.10.2023 um 12:55 schrieb Richard Zowalla:
>>>>> Cool. Thanks for going down the rabbit hole ;-)
>>>>>
>>>>> Given that there is a lot of information/benchmarks etc. collected in
>>>>> this thread, what do you think about opening an issue in Tomcat with
>>>>> a reference to this thread?
>>>>>
>>>>> They might take a look.
>>>>>
>>>>> Gruß
>>>>> Richard
>>>>>
>>>>> Am 28. Oktober 2023 12:22:12 MESZ schrieb Jens Zurawski
>>>>> <jz...@diegurus.de>:
>>>>>> For all who are interested I've made some quick benchmarks, to see
>>>>>> the impact.
>>>>>> I've taken one of my biggest views which is making a small AJAX poll
>>>>>> on the same view every 30s to be able to get some balanced average
>>>>>> values for the response times. The payload of the poll is small
>>>>>> (around 500 to 1000 bytes) but because it's the same view, the
>>>>>> component tree is big.
>>>>>>
>>>>>> Windows (same machine localhost, it is an 8 core Ryzen-7 with 64GB
>>>>>> of RAM and SSD, so the machine is not the limiting factor)
>>>>>> ====================================================
>>>>>> ~ 12000 ms DevMode plain
>>>>>> ~ 600 ms DevMode with <Resources archiveIndexStrategy="bloom"
>>>>>> allowLinking="true"/>
>>>>>> ~ 100 ms ProdMode plain
>>>>>> ~ 100 ms ProdMode with <Resources archiveIndexStrategy="bloom"
>>>>>> allowLinking="true"/>
>>>>>>
>>>>>> Linux (customer site over Internet VPN, it is a 6 core Intel i5 with
>>>>>> 16GB of RAM and SSD)
>>>>>> ====================================================
>>>>>> ~ 700 ms DevMode plain
>>>>>> ~ 350 ms DevMode with <Resources archiveIndexStrategy="bloom"
>>>>>> allowLinking="true"/>
>>>>>> ~ 130 ms ProdMode plain
>>>>>> ~ 130 ms ProdMode with <Resources archiveIndexStrategy="bloom"
>>>>>> allowLinking="true"/>
>>>>>>
>>>>>> No, the 12000 ms is _not_ a typo ;-)
>>>>>>
>>>>>> My conclusions from these figures are:
>>>>>> - ProdMode is not affected by this issue
>>>>>> - On Windows the problem is very significant and renders the DevMode
>>>>>> unusable
>>>>>> - On Linux the problem is not that dramatic, so one can say "well,
>>>>>> it's dev mode, deal with it", but it is also affected in doubling
>>>>>> the response times. Maybe worth enough to also look into this issue
>>>>>> even if one is Linux only.
>>>>>>
>>>>>> cu
>>>>>> Jens
>>>>>>
>>>>>>
>>>>>>
>>>>>> Am 28.10.2023 um 11:43 schrieb Jens Zurawski:
>>>>>>> I can confirm, that this setting also solves the problem of the
>>>>>>> slow dev mode. Thank you for the tip.
>>>>>>>
>>>>>>> So, either this or the canonCache solves this issue for me on my
>>>>>>> development environment. But, of course, both of them should be
>>>>>>> avoided on production systems. The chances that they slip from dev
>>>>>>> to prod by accident may be small (if one is using well known "good"
>>>>>>> development strategies and methods, like staging, testing Q&A,
>>>>>>> reviews, and the like) but still possible.
>>>>>>> And the fact, that it seems to only really matter in JSF dev mode
>>>>>>> is curious at the least.
>>>>>>>
>>>>>>> Am 28.10.2023 um 08:24 schrieb Vicente Rossello:
>>>>>>>> Sorry, that was not correct.
>>>>>>>>
>>>>>>>> The allowLinking is at the resources level. So, in the
>>>>>>>> META-INF/context.xml
>>>>>>>> file:
>>>>>>>>
>>>>>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>>>>> <Context allowCasualMultipartParsing="true" reloadable="false"
>>>>>>>>
>>> containerSciFilter="org.apache.tomcat.websocket.server.WsSci|org.apache.jasper.servlet.JasperInitializer"
>>>
>>>>>>>> parallelAnnotationScanning="true"
>>>>>>>> clearReferencesThreadLocals="false"
>>>>>>>> skipMemoryLeakChecksOnJvmShutdown="true">
>>>>>>>> <Manager pathname=""/>
>>>>>>>>
>>>>>>>> <Resources archiveIndexStrategy="bloom" allowLinking="true"/>
>>>>>>>> </Context>
>>>>>>>>
>>>>>>>>
>>>>>>>> BTW, the slowness is something between checking the canonicalPath
>>> too
>>>>>>>> many times and jdk12 that removed those canonCache by default. See
>>>>>>>> https://bugs.openjdk.org/browse/JDK-8258036
>>>>>>>>
>>>>>>>>
>>>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Vicente Rossello <co...@gmail.com>.
https://bz.apache.org/bugzilla/show_bug.cgi?id=68054
Let's see how it works
On Tue, Oct 31, 2023 at 3:04 PM Vicente Rossello <co...@gmail.com>
wrote:
> I will try to open a bug. An obvious fix is to only make 1 call to
> getCannonicalPath for the same file. That would halve the time.
>
> After that I guess is tomcat team decission.... maybe emitting a warning
> that allowLinking=false in windows is very slow for jsf development ... or
> they may have more ideas
>
> El mar, 31 oct 2023, 14:20, Jens Zurawski <jz...@diegurus.de> escribió:
>
>> Ok, after 4 hours of bisecting, checking out, building and testing (I've
>> needed three bisect sessions, because the first one wasn't successful
>> due to build failures of some revisions and I had to manually find some
>> other revisions to bisect) I can definitely identify the commit which
>> introduces the DevMode performance issue. It is:
>>
>> 0325d91 TOMEE-3773 - Update Tomcat to 9.0.50
>>
>> I've cross checked with the immediate commit before (277fc71) and it's
>> clear:
>> 277fc71 good
>> 0325d91 bad
>>
>> So, indeed, tomcat is the root cause of the problem. But, I don't think
>> it will be successful to create an issue in tomcat. A far as I can see
>> in the diffs of tomcat 9.0.48 to 9.0.50, I assume, it already was a bug
>> fix in /java/org/apache/catalina/webresources/DirResourceSet.java. In
>> that class new code was inserted to skip symbolic links in
>> ServletContext.getResourcePaths(), which, I think, is important as a
>> security measure. Therefore, most likely they will not be willing to
>> revert that (I wouldn't recommend that, either)
>>
>> Long story short: It's not a bug it's a feature ;-)
>>
>> I guess, JSF in DevMode is doing a lot more calls to getResourcePaths()
>> hence suffering from this fix. Because ProdMode is not affected by this
>> I can live with it. Development environments are special in any case,
>> and to allow the dev tomcat following symlinks in that environment is
>> ok, from my pov. Just enable the allowLinking="true" and it works.
>>
>> At least my "academic problem" is solved and my "inner monk" is
>> satisfied and I don't think I will walk down this route any further.
>>
>> Many thanks to all of you who have helped me on this issue.
>>
>> cu
>> Jens
>>
>> Am 28.10.2023 um 15:17 schrieb Jens Zurawski:
>> > Yes, this may be one of the next steps here. But, I'm afraid, I have
>> > not enough info for them right now.
>> > Neither do I have a small test case to reproduce the problem, nor can
>> > I point them exact enough to what the problem might be. Also, maybe
>> > it's only coincidence, and the real problem is somewhere inside
>> > myfaces. The fact that ProdMode is not affected at all (at least not
>> > enough to measure any difference) and it only arises in JSF DevMode
>> > could point in that direction.
>> >
>> > So, before I go and open an issue on Tomcat, I want to try to narrow
>> > down it more. At least to the point where I can say: look, this commit
>> > in TomEE introduced it. And even better: here is a small test case to
>> > reproduce it.
>> >
>> > cu
>> > Jens
>> >
>> >
>> > Am 28.10.2023 um 12:55 schrieb Richard Zowalla:
>> >> Cool. Thanks for going down the rabbit hole ;-)
>> >>
>> >> Given that there is a lot of information/benchmarks etc. collected in
>> >> this thread, what do you think about opening an issue in Tomcat with
>> >> a reference to this thread?
>> >>
>> >> They might take a look.
>> >>
>> >> Gruß
>> >> Richard
>> >>
>> >> Am 28. Oktober 2023 12:22:12 MESZ schrieb Jens Zurawski
>> >> <jz...@diegurus.de>:
>> >>> For all who are interested I've made some quick benchmarks, to see
>> >>> the impact.
>> >>> I've taken one of my biggest views which is making a small AJAX poll
>> >>> on the same view every 30s to be able to get some balanced average
>> >>> values for the response times. The payload of the poll is small
>> >>> (around 500 to 1000 bytes) but because it's the same view, the
>> >>> component tree is big.
>> >>>
>> >>> Windows (same machine localhost, it is an 8 core Ryzen-7 with 64GB
>> >>> of RAM and SSD, so the machine is not the limiting factor)
>> >>> ====================================================
>> >>> ~ 12000 ms DevMode plain
>> >>> ~ 600 ms DevMode with <Resources archiveIndexStrategy="bloom"
>> >>> allowLinking="true"/>
>> >>> ~ 100 ms ProdMode plain
>> >>> ~ 100 ms ProdMode with <Resources archiveIndexStrategy="bloom"
>> >>> allowLinking="true"/>
>> >>>
>> >>> Linux (customer site over Internet VPN, it is a 6 core Intel i5 with
>> >>> 16GB of RAM and SSD)
>> >>> ====================================================
>> >>> ~ 700 ms DevMode plain
>> >>> ~ 350 ms DevMode with <Resources archiveIndexStrategy="bloom"
>> >>> allowLinking="true"/>
>> >>> ~ 130 ms ProdMode plain
>> >>> ~ 130 ms ProdMode with <Resources archiveIndexStrategy="bloom"
>> >>> allowLinking="true"/>
>> >>>
>> >>> No, the 12000 ms is _not_ a typo ;-)
>> >>>
>> >>> My conclusions from these figures are:
>> >>> - ProdMode is not affected by this issue
>> >>> - On Windows the problem is very significant and renders the DevMode
>> >>> unusable
>> >>> - On Linux the problem is not that dramatic, so one can say "well,
>> >>> it's dev mode, deal with it", but it is also affected in doubling
>> >>> the response times. Maybe worth enough to also look into this issue
>> >>> even if one is Linux only.
>> >>>
>> >>> cu
>> >>> Jens
>> >>>
>> >>>
>> >>>
>> >>> Am 28.10.2023 um 11:43 schrieb Jens Zurawski:
>> >>>> I can confirm, that this setting also solves the problem of the
>> >>>> slow dev mode. Thank you for the tip.
>> >>>>
>> >>>> So, either this or the canonCache solves this issue for me on my
>> >>>> development environment. But, of course, both of them should be
>> >>>> avoided on production systems. The chances that they slip from dev
>> >>>> to prod by accident may be small (if one is using well known "good"
>> >>>> development strategies and methods, like staging, testing Q&A,
>> >>>> reviews, and the like) but still possible.
>> >>>> And the fact, that it seems to only really matter in JSF dev mode
>> >>>> is curious at the least.
>> >>>>
>> >>>> Am 28.10.2023 um 08:24 schrieb Vicente Rossello:
>> >>>>> Sorry, that was not correct.
>> >>>>>
>> >>>>> The allowLinking is at the resources level. So, in the
>> >>>>> META-INF/context.xml
>> >>>>> file:
>> >>>>>
>> >>>>> <?xml version="1.0" encoding="UTF-8"?>
>> >>>>> <Context allowCasualMultipartParsing="true" reloadable="false"
>> >>>>>
>> containerSciFilter="org.apache.tomcat.websocket.server.WsSci|org.apache.jasper.servlet.JasperInitializer"
>>
>> >>>>>
>> >>>>> parallelAnnotationScanning="true"
>> >>>>> clearReferencesThreadLocals="false"
>> >>>>> skipMemoryLeakChecksOnJvmShutdown="true">
>> >>>>> <Manager pathname=""/>
>> >>>>>
>> >>>>> <Resources archiveIndexStrategy="bloom" allowLinking="true"/>
>> >>>>> </Context>
>> >>>>>
>> >>>>>
>> >>>>> BTW, the slowness is something between checking the canonicalPath
>> too
>> >>>>> many times and jdk12 that removed those canonCache by default. See
>> >>>>> https://bugs.openjdk.org/browse/JDK-8258036
>> >>>>>
>> >>>>>
>> >
>>
>>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Vicente Rossello <co...@gmail.com>.
I will try to open a bug. An obvious fix is to only make 1 call to
getCannonicalPath for the same file. That would halve the time.
After that I guess is tomcat team decission.... maybe emitting a warning
that allowLinking=false in windows is very slow for jsf development ... or
they may have more ideas
El mar, 31 oct 2023, 14:20, Jens Zurawski <jz...@diegurus.de> escribió:
> Ok, after 4 hours of bisecting, checking out, building and testing (I've
> needed three bisect sessions, because the first one wasn't successful
> due to build failures of some revisions and I had to manually find some
> other revisions to bisect) I can definitely identify the commit which
> introduces the DevMode performance issue. It is:
>
> 0325d91 TOMEE-3773 - Update Tomcat to 9.0.50
>
> I've cross checked with the immediate commit before (277fc71) and it's
> clear:
> 277fc71 good
> 0325d91 bad
>
> So, indeed, tomcat is the root cause of the problem. But, I don't think
> it will be successful to create an issue in tomcat. A far as I can see
> in the diffs of tomcat 9.0.48 to 9.0.50, I assume, it already was a bug
> fix in /java/org/apache/catalina/webresources/DirResourceSet.java. In
> that class new code was inserted to skip symbolic links in
> ServletContext.getResourcePaths(), which, I think, is important as a
> security measure. Therefore, most likely they will not be willing to
> revert that (I wouldn't recommend that, either)
>
> Long story short: It's not a bug it's a feature ;-)
>
> I guess, JSF in DevMode is doing a lot more calls to getResourcePaths()
> hence suffering from this fix. Because ProdMode is not affected by this
> I can live with it. Development environments are special in any case,
> and to allow the dev tomcat following symlinks in that environment is
> ok, from my pov. Just enable the allowLinking="true" and it works.
>
> At least my "academic problem" is solved and my "inner monk" is
> satisfied and I don't think I will walk down this route any further.
>
> Many thanks to all of you who have helped me on this issue.
>
> cu
> Jens
>
> Am 28.10.2023 um 15:17 schrieb Jens Zurawski:
> > Yes, this may be one of the next steps here. But, I'm afraid, I have
> > not enough info for them right now.
> > Neither do I have a small test case to reproduce the problem, nor can
> > I point them exact enough to what the problem might be. Also, maybe
> > it's only coincidence, and the real problem is somewhere inside
> > myfaces. The fact that ProdMode is not affected at all (at least not
> > enough to measure any difference) and it only arises in JSF DevMode
> > could point in that direction.
> >
> > So, before I go and open an issue on Tomcat, I want to try to narrow
> > down it more. At least to the point where I can say: look, this commit
> > in TomEE introduced it. And even better: here is a small test case to
> > reproduce it.
> >
> > cu
> > Jens
> >
> >
> > Am 28.10.2023 um 12:55 schrieb Richard Zowalla:
> >> Cool. Thanks for going down the rabbit hole ;-)
> >>
> >> Given that there is a lot of information/benchmarks etc. collected in
> >> this thread, what do you think about opening an issue in Tomcat with
> >> a reference to this thread?
> >>
> >> They might take a look.
> >>
> >> Gruß
> >> Richard
> >>
> >> Am 28. Oktober 2023 12:22:12 MESZ schrieb Jens Zurawski
> >> <jz...@diegurus.de>:
> >>> For all who are interested I've made some quick benchmarks, to see
> >>> the impact.
> >>> I've taken one of my biggest views which is making a small AJAX poll
> >>> on the same view every 30s to be able to get some balanced average
> >>> values for the response times. The payload of the poll is small
> >>> (around 500 to 1000 bytes) but because it's the same view, the
> >>> component tree is big.
> >>>
> >>> Windows (same machine localhost, it is an 8 core Ryzen-7 with 64GB
> >>> of RAM and SSD, so the machine is not the limiting factor)
> >>> ====================================================
> >>> ~ 12000 ms DevMode plain
> >>> ~ 600 ms DevMode with <Resources archiveIndexStrategy="bloom"
> >>> allowLinking="true"/>
> >>> ~ 100 ms ProdMode plain
> >>> ~ 100 ms ProdMode with <Resources archiveIndexStrategy="bloom"
> >>> allowLinking="true"/>
> >>>
> >>> Linux (customer site over Internet VPN, it is a 6 core Intel i5 with
> >>> 16GB of RAM and SSD)
> >>> ====================================================
> >>> ~ 700 ms DevMode plain
> >>> ~ 350 ms DevMode with <Resources archiveIndexStrategy="bloom"
> >>> allowLinking="true"/>
> >>> ~ 130 ms ProdMode plain
> >>> ~ 130 ms ProdMode with <Resources archiveIndexStrategy="bloom"
> >>> allowLinking="true"/>
> >>>
> >>> No, the 12000 ms is _not_ a typo ;-)
> >>>
> >>> My conclusions from these figures are:
> >>> - ProdMode is not affected by this issue
> >>> - On Windows the problem is very significant and renders the DevMode
> >>> unusable
> >>> - On Linux the problem is not that dramatic, so one can say "well,
> >>> it's dev mode, deal with it", but it is also affected in doubling
> >>> the response times. Maybe worth enough to also look into this issue
> >>> even if one is Linux only.
> >>>
> >>> cu
> >>> Jens
> >>>
> >>>
> >>>
> >>> Am 28.10.2023 um 11:43 schrieb Jens Zurawski:
> >>>> I can confirm, that this setting also solves the problem of the
> >>>> slow dev mode. Thank you for the tip.
> >>>>
> >>>> So, either this or the canonCache solves this issue for me on my
> >>>> development environment. But, of course, both of them should be
> >>>> avoided on production systems. The chances that they slip from dev
> >>>> to prod by accident may be small (if one is using well known "good"
> >>>> development strategies and methods, like staging, testing Q&A,
> >>>> reviews, and the like) but still possible.
> >>>> And the fact, that it seems to only really matter in JSF dev mode
> >>>> is curious at the least.
> >>>>
> >>>> Am 28.10.2023 um 08:24 schrieb Vicente Rossello:
> >>>>> Sorry, that was not correct.
> >>>>>
> >>>>> The allowLinking is at the resources level. So, in the
> >>>>> META-INF/context.xml
> >>>>> file:
> >>>>>
> >>>>> <?xml version="1.0" encoding="UTF-8"?>
> >>>>> <Context allowCasualMultipartParsing="true" reloadable="false"
> >>>>>
> containerSciFilter="org.apache.tomcat.websocket.server.WsSci|org.apache.jasper.servlet.JasperInitializer"
>
> >>>>>
> >>>>> parallelAnnotationScanning="true"
> >>>>> clearReferencesThreadLocals="false"
> >>>>> skipMemoryLeakChecksOnJvmShutdown="true">
> >>>>> <Manager pathname=""/>
> >>>>>
> >>>>> <Resources archiveIndexStrategy="bloom" allowLinking="true"/>
> >>>>> </Context>
> >>>>>
> >>>>>
> >>>>> BTW, the slowness is something between checking the canonicalPath too
> >>>>> many times and jdk12 that removed those canonCache by default. See
> >>>>> https://bugs.openjdk.org/browse/JDK-8258036
> >>>>>
> >>>>>
> >
>
>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Jens Zurawski <jz...@diegurus.de>.
Ok, after 4 hours of bisecting, checking out, building and testing (I've
needed three bisect sessions, because the first one wasn't successful
due to build failures of some revisions and I had to manually find some
other revisions to bisect) I can definitely identify the commit which
introduces the DevMode performance issue. It is:
0325d91 TOMEE-3773 - Update Tomcat to 9.0.50
I've cross checked with the immediate commit before (277fc71) and it's
clear:
277fc71 good
0325d91 bad
So, indeed, tomcat is the root cause of the problem. But, I don't think
it will be successful to create an issue in tomcat. A far as I can see
in the diffs of tomcat 9.0.48 to 9.0.50, I assume, it already was a bug
fix in /java/org/apache/catalina/webresources/DirResourceSet.java. In
that class new code was inserted to skip symbolic links in
ServletContext.getResourcePaths(), which, I think, is important as a
security measure. Therefore, most likely they will not be willing to
revert that (I wouldn't recommend that, either)
Long story short: It's not a bug it's a feature ;-)
I guess, JSF in DevMode is doing a lot more calls to getResourcePaths()
hence suffering from this fix. Because ProdMode is not affected by this
I can live with it. Development environments are special in any case,
and to allow the dev tomcat following symlinks in that environment is
ok, from my pov. Just enable the allowLinking="true" and it works.
At least my "academic problem" is solved and my "inner monk" is
satisfied and I don't think I will walk down this route any further.
Many thanks to all of you who have helped me on this issue.
cu
Jens
Am 28.10.2023 um 15:17 schrieb Jens Zurawski:
> Yes, this may be one of the next steps here. But, I'm afraid, I have
> not enough info for them right now.
> Neither do I have a small test case to reproduce the problem, nor can
> I point them exact enough to what the problem might be. Also, maybe
> it's only coincidence, and the real problem is somewhere inside
> myfaces. The fact that ProdMode is not affected at all (at least not
> enough to measure any difference) and it only arises in JSF DevMode
> could point in that direction.
>
> So, before I go and open an issue on Tomcat, I want to try to narrow
> down it more. At least to the point where I can say: look, this commit
> in TomEE introduced it. And even better: here is a small test case to
> reproduce it.
>
> cu
> Jens
>
>
> Am 28.10.2023 um 12:55 schrieb Richard Zowalla:
>> Cool. Thanks for going down the rabbit hole ;-)
>>
>> Given that there is a lot of information/benchmarks etc. collected in
>> this thread, what do you think about opening an issue in Tomcat with
>> a reference to this thread?
>>
>> They might take a look.
>>
>> Gruß
>> Richard
>>
>> Am 28. Oktober 2023 12:22:12 MESZ schrieb Jens Zurawski
>> <jz...@diegurus.de>:
>>> For all who are interested I've made some quick benchmarks, to see
>>> the impact.
>>> I've taken one of my biggest views which is making a small AJAX poll
>>> on the same view every 30s to be able to get some balanced average
>>> values for the response times. The payload of the poll is small
>>> (around 500 to 1000 bytes) but because it's the same view, the
>>> component tree is big.
>>>
>>> Windows (same machine localhost, it is an 8 core Ryzen-7 with 64GB
>>> of RAM and SSD, so the machine is not the limiting factor)
>>> ====================================================
>>> ~ 12000 ms DevMode plain
>>> ~ 600 ms DevMode with <Resources archiveIndexStrategy="bloom"
>>> allowLinking="true"/>
>>> ~ 100 ms ProdMode plain
>>> ~ 100 ms ProdMode with <Resources archiveIndexStrategy="bloom"
>>> allowLinking="true"/>
>>>
>>> Linux (customer site over Internet VPN, it is a 6 core Intel i5 with
>>> 16GB of RAM and SSD)
>>> ====================================================
>>> ~ 700 ms DevMode plain
>>> ~ 350 ms DevMode with <Resources archiveIndexStrategy="bloom"
>>> allowLinking="true"/>
>>> ~ 130 ms ProdMode plain
>>> ~ 130 ms ProdMode with <Resources archiveIndexStrategy="bloom"
>>> allowLinking="true"/>
>>>
>>> No, the 12000 ms is _not_ a typo ;-)
>>>
>>> My conclusions from these figures are:
>>> - ProdMode is not affected by this issue
>>> - On Windows the problem is very significant and renders the DevMode
>>> unusable
>>> - On Linux the problem is not that dramatic, so one can say "well,
>>> it's dev mode, deal with it", but it is also affected in doubling
>>> the response times. Maybe worth enough to also look into this issue
>>> even if one is Linux only.
>>>
>>> cu
>>> Jens
>>>
>>>
>>>
>>> Am 28.10.2023 um 11:43 schrieb Jens Zurawski:
>>>> I can confirm, that this setting also solves the problem of the
>>>> slow dev mode. Thank you for the tip.
>>>>
>>>> So, either this or the canonCache solves this issue for me on my
>>>> development environment. But, of course, both of them should be
>>>> avoided on production systems. The chances that they slip from dev
>>>> to prod by accident may be small (if one is using well known "good"
>>>> development strategies and methods, like staging, testing Q&A,
>>>> reviews, and the like) but still possible.
>>>> And the fact, that it seems to only really matter in JSF dev mode
>>>> is curious at the least.
>>>>
>>>> Am 28.10.2023 um 08:24 schrieb Vicente Rossello:
>>>>> Sorry, that was not correct.
>>>>>
>>>>> The allowLinking is at the resources level. So, in the
>>>>> META-INF/context.xml
>>>>> file:
>>>>>
>>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>> <Context allowCasualMultipartParsing="true" reloadable="false"
>>>>> containerSciFilter="org.apache.tomcat.websocket.server.WsSci|org.apache.jasper.servlet.JasperInitializer"
>>>>>
>>>>> parallelAnnotationScanning="true"
>>>>> clearReferencesThreadLocals="false"
>>>>> skipMemoryLeakChecksOnJvmShutdown="true">
>>>>> <Manager pathname=""/>
>>>>>
>>>>> <Resources archiveIndexStrategy="bloom" allowLinking="true"/>
>>>>> </Context>
>>>>>
>>>>>
>>>>> BTW, the slowness is something between checking the canonicalPath too
>>>>> many times and jdk12 that removed those canonCache by default. See
>>>>> https://bugs.openjdk.org/browse/JDK-8258036
>>>>>
>>>>>
>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Jens Zurawski <jz...@diegurus.de>.
Yes, this may be one of the next steps here. But, I'm afraid, I have not
enough info for them right now.
Neither do I have a small test case to reproduce the problem, nor can I
point them exact enough to what the problem might be. Also, maybe it's
only coincidence, and the real problem is somewhere inside myfaces. The
fact that ProdMode is not affected at all (at least not enough to
measure any difference) and it only arises in JSF DevMode could point in
that direction.
So, before I go and open an issue on Tomcat, I want to try to narrow
down it more. At least to the point where I can say: look, this commit
in TomEE introduced it. And even better: here is a small test case to
reproduce it.
cu
Jens
Am 28.10.2023 um 12:55 schrieb Richard Zowalla:
> Cool. Thanks for going down the rabbit hole ;-)
>
> Given that there is a lot of information/benchmarks etc. collected in this thread, what do you think about opening an issue in Tomcat with a reference to this thread?
>
> They might take a look.
>
> Gruß
> Richard
>
> Am 28. Oktober 2023 12:22:12 MESZ schrieb Jens Zurawski <jz...@diegurus.de>:
>> For all who are interested I've made some quick benchmarks, to see the impact.
>> I've taken one of my biggest views which is making a small AJAX poll on the same view every 30s to be able to get some balanced average values for the response times. The payload of the poll is small (around 500 to 1000 bytes) but because it's the same view, the component tree is big.
>>
>> Windows (same machine localhost, it is an 8 core Ryzen-7 with 64GB of RAM and SSD, so the machine is not the limiting factor)
>> ====================================================
>> ~ 12000 ms DevMode plain
>> ~ 600 ms DevMode with <Resources archiveIndexStrategy="bloom" allowLinking="true"/>
>> ~ 100 ms ProdMode plain
>> ~ 100 ms ProdMode with <Resources archiveIndexStrategy="bloom" allowLinking="true"/>
>>
>> Linux (customer site over Internet VPN, it is a 6 core Intel i5 with 16GB of RAM and SSD)
>> ====================================================
>> ~ 700 ms DevMode plain
>> ~ 350 ms DevMode with <Resources archiveIndexStrategy="bloom" allowLinking="true"/>
>> ~ 130 ms ProdMode plain
>> ~ 130 ms ProdMode with <Resources archiveIndexStrategy="bloom" allowLinking="true"/>
>>
>> No, the 12000 ms is _not_ a typo ;-)
>>
>> My conclusions from these figures are:
>> - ProdMode is not affected by this issue
>> - On Windows the problem is very significant and renders the DevMode unusable
>> - On Linux the problem is not that dramatic, so one can say "well, it's dev mode, deal with it", but it is also affected in doubling the response times. Maybe worth enough to also look into this issue even if one is Linux only.
>>
>> cu
>> Jens
>>
>>
>>
>> Am 28.10.2023 um 11:43 schrieb Jens Zurawski:
>>> I can confirm, that this setting also solves the problem of the slow dev mode. Thank you for the tip.
>>>
>>> So, either this or the canonCache solves this issue for me on my development environment. But, of course, both of them should be avoided on production systems. The chances that they slip from dev to prod by accident may be small (if one is using well known "good" development strategies and methods, like staging, testing Q&A, reviews, and the like) but still possible.
>>> And the fact, that it seems to only really matter in JSF dev mode is curious at the least.
>>>
>>> Am 28.10.2023 um 08:24 schrieb Vicente Rossello:
>>>> Sorry, that was not correct.
>>>>
>>>> The allowLinking is at the resources level. So, in the META-INF/context.xml
>>>> file:
>>>>
>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>> <Context allowCasualMultipartParsing="true" reloadable="false"
>>>> containerSciFilter="org.apache.tomcat.websocket.server.WsSci|org.apache.jasper.servlet.JasperInitializer"
>>>> parallelAnnotationScanning="true"
>>>> clearReferencesThreadLocals="false"
>>>> skipMemoryLeakChecksOnJvmShutdown="true">
>>>> <Manager pathname=""/>
>>>>
>>>> <Resources archiveIndexStrategy="bloom" allowLinking="true"/>
>>>> </Context>
>>>>
>>>>
>>>> BTW, the slowness is something between checking the canonicalPath too
>>>> many times and jdk12 that removed those canonCache by default. See
>>>> https://bugs.openjdk.org/browse/JDK-8258036
>>>>
>>>>
--
jens zurawski
diegurus - zurawski zurawski poppl rohland GbR
juister straße 3
65199 wiesbaden
kaspersweg 7b
26131 oldenburg
internet http://www.diegurus.de
tel +49(0)611 72437966
CONFIDENTIALITY NOTICE: This e-mail message is intended only for the
person or entity to which it is addressed and may contain confidential
and/or privileged material. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient,
please contact the sender by reply e-mail and destroy all copies of the
original message. If you are the intended recipient but do not wish to
receive communications through this medium, please so advise the sender
immediately.
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Richard Zowalla <ri...@zowalla.com>.
Cool. Thanks for going down the rabbit hole ;-)
Given that there is a lot of information/benchmarks etc. collected in this thread, what do you think about opening an issue in Tomcat with a reference to this thread?
They might take a look.
Gruß
Richard
Am 28. Oktober 2023 12:22:12 MESZ schrieb Jens Zurawski <jz...@diegurus.de>:
>For all who are interested I've made some quick benchmarks, to see the impact.
>I've taken one of my biggest views which is making a small AJAX poll on the same view every 30s to be able to get some balanced average values for the response times. The payload of the poll is small (around 500 to 1000 bytes) but because it's the same view, the component tree is big.
>
>Windows (same machine localhost, it is an 8 core Ryzen-7 with 64GB of RAM and SSD, so the machine is not the limiting factor)
>====================================================
>~ 12000 ms DevMode plain
>~ 600 ms DevMode with <Resources archiveIndexStrategy="bloom" allowLinking="true"/>
>~ 100 ms ProdMode plain
>~ 100 ms ProdMode with <Resources archiveIndexStrategy="bloom" allowLinking="true"/>
>
>Linux (customer site over Internet VPN, it is a 6 core Intel i5 with 16GB of RAM and SSD)
>====================================================
>~ 700 ms DevMode plain
>~ 350 ms DevMode with <Resources archiveIndexStrategy="bloom" allowLinking="true"/>
>~ 130 ms ProdMode plain
>~ 130 ms ProdMode with <Resources archiveIndexStrategy="bloom" allowLinking="true"/>
>
>No, the 12000 ms is _not_ a typo ;-)
>
>My conclusions from these figures are:
>- ProdMode is not affected by this issue
>- On Windows the problem is very significant and renders the DevMode unusable
>- On Linux the problem is not that dramatic, so one can say "well, it's dev mode, deal with it", but it is also affected in doubling the response times. Maybe worth enough to also look into this issue even if one is Linux only.
>
>cu
>Jens
>
>
>
>Am 28.10.2023 um 11:43 schrieb Jens Zurawski:
>> I can confirm, that this setting also solves the problem of the slow dev mode. Thank you for the tip.
>>
>> So, either this or the canonCache solves this issue for me on my development environment. But, of course, both of them should be avoided on production systems. The chances that they slip from dev to prod by accident may be small (if one is using well known "good" development strategies and methods, like staging, testing Q&A, reviews, and the like) but still possible.
>> And the fact, that it seems to only really matter in JSF dev mode is curious at the least.
>>
>> Am 28.10.2023 um 08:24 schrieb Vicente Rossello:
>>> Sorry, that was not correct.
>>>
>>> The allowLinking is at the resources level. So, in the META-INF/context.xml
>>> file:
>>>
>>> <?xml version="1.0" encoding="UTF-8"?>
>>> <Context allowCasualMultipartParsing="true" reloadable="false"
>>> containerSciFilter="org.apache.tomcat.websocket.server.WsSci|org.apache.jasper.servlet.JasperInitializer"
>>> parallelAnnotationScanning="true"
>>> clearReferencesThreadLocals="false"
>>> skipMemoryLeakChecksOnJvmShutdown="true">
>>> <Manager pathname=""/>
>>>
>>> <Resources archiveIndexStrategy="bloom" allowLinking="true"/>
>>> </Context>
>>>
>>>
>>> BTW, the slowness is something between checking the canonicalPath too
>>> many times and jdk12 that removed those canonCache by default. See
>>> https://bugs.openjdk.org/browse/JDK-8258036
>>>
>>>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Jens Zurawski <jz...@diegurus.de>.
For all who are interested I've made some quick benchmarks, to see the
impact.
I've taken one of my biggest views which is making a small AJAX poll on
the same view every 30s to be able to get some balanced average values
for the response times. The payload of the poll is small (around 500 to
1000 bytes) but because it's the same view, the component tree is big.
Windows (same machine localhost, it is an 8 core Ryzen-7 with 64GB of
RAM and SSD, so the machine is not the limiting factor)
====================================================
~ 12000 ms DevMode plain
~ 600 ms DevMode with <Resources archiveIndexStrategy="bloom"
allowLinking="true"/>
~ 100 ms ProdMode plain
~ 100 ms ProdMode with <Resources archiveIndexStrategy="bloom"
allowLinking="true"/>
Linux (customer site over Internet VPN, it is a 6 core Intel i5 with
16GB of RAM and SSD)
====================================================
~ 700 ms DevMode plain
~ 350 ms DevMode with <Resources archiveIndexStrategy="bloom"
allowLinking="true"/>
~ 130 ms ProdMode plain
~ 130 ms ProdMode with <Resources archiveIndexStrategy="bloom"
allowLinking="true"/>
No, the 12000 ms is _not_ a typo ;-)
My conclusions from these figures are:
- ProdMode is not affected by this issue
- On Windows the problem is very significant and renders the DevMode
unusable
- On Linux the problem is not that dramatic, so one can say "well, it's
dev mode, deal with it", but it is also affected in doubling the
response times. Maybe worth enough to also look into this issue even if
one is Linux only.
cu
Jens
Am 28.10.2023 um 11:43 schrieb Jens Zurawski:
> I can confirm, that this setting also solves the problem of the slow
> dev mode. Thank you for the tip.
>
> So, either this or the canonCache solves this issue for me on my
> development environment. But, of course, both of them should be
> avoided on production systems. The chances that they slip from dev to
> prod by accident may be small (if one is using well known "good"
> development strategies and methods, like staging, testing Q&A,
> reviews, and the like) but still possible.
> And the fact, that it seems to only really matter in JSF dev mode is
> curious at the least.
>
> Am 28.10.2023 um 08:24 schrieb Vicente Rossello:
>> Sorry, that was not correct.
>>
>> The allowLinking is at the resources level. So, in the
>> META-INF/context.xml
>> file:
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>> <Context allowCasualMultipartParsing="true" reloadable="false"
>> containerSciFilter="org.apache.tomcat.websocket.server.WsSci|org.apache.jasper.servlet.JasperInitializer"
>>
>> parallelAnnotationScanning="true"
>> clearReferencesThreadLocals="false"
>> skipMemoryLeakChecksOnJvmShutdown="true">
>> <Manager pathname=""/>
>>
>> <Resources archiveIndexStrategy="bloom" allowLinking="true"/>
>> </Context>
>>
>>
>> BTW, the slowness is something between checking the canonicalPath too
>> many times and jdk12 that removed those canonCache by default. See
>> https://bugs.openjdk.org/browse/JDK-8258036
>>
>>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Jens Zurawski <jz...@diegurus.de>.
I can confirm, that this setting also solves the problem of the slow dev
mode. Thank you for the tip.
So, either this or the canonCache solves this issue for me on my
development environment. But, of course, both of them should be avoided
on production systems. The chances that they slip from dev to prod by
accident may be small (if one is using well known "good" development
strategies and methods, like staging, testing Q&A, reviews, and the
like) but still possible.
And the fact, that it seems to only really matter in JSF dev mode is
curious at the least.
Am 28.10.2023 um 08:24 schrieb Vicente Rossello:
> Sorry, that was not correct.
>
> The allowLinking is at the resources level. So, in the META-INF/context.xml
> file:
>
> <?xml version="1.0" encoding="UTF-8"?>
> <Context allowCasualMultipartParsing="true" reloadable="false"
> containerSciFilter="org.apache.tomcat.websocket.server.WsSci|org.apache.jasper.servlet.JasperInitializer"
> parallelAnnotationScanning="true"
> clearReferencesThreadLocals="false"
> skipMemoryLeakChecksOnJvmShutdown="true">
> <Manager pathname=""/>
>
> <Resources archiveIndexStrategy="bloom" allowLinking="true"/>
> </Context>
>
>
> BTW, the slowness is something between checking the canonicalPath too
> many times and jdk12 that removed those canonCache by default. See
> https://bugs.openjdk.org/browse/JDK-8258036
>
>
> On Sat, Oct 28, 2023 at 8:07 AM Vicente Rossello <co...@gmail.com>
> wrote:
>
>> Hi,
>>
>> It is a tomcat problem. The real problem is in DirResourcesSet. It is
>> calling a lot of times File.getCanonicalPath(). This method is very slow in
>> windows.
>>
>> What I did is extending the DirResourcesSet to skip processing the
>> getCanonicalPath method so many times, but I really didn't understand the
>> problem.
>>
>> A simpler and better solution is to set allowLinking="true" in the
>> context.xml file. So tomcat does not validate the canonicalPath every
>> single time, which is just too much.
>>
>> Best regards,
>> Vicente
>>
>> On Fri, Oct 27, 2023 at 5:51 PM Jens Zurawski <jz...@diegurus.de> wrote:
>>
>>> My client is the last Windows System in my little universe. Due to some
>>> constraints with my customers I have to deal with it for the time being.
>>>
>>> The switches, Vicente has provided, solved the problem for me so far.
>>> But my "inner monk" is still not satisfied, because practical no one
>>> knows of these switches, and after all, TomEE should do a good job on
>>> Windows, too. And because 8.0.6 does a good job, and 8.0.8 breaks it to
>>> some degree, we should at least try to figure out, what exactly causes
>>> this break. Maybe there is an easy way to fix it, then it should be
>>> fixed IMHO.
>>>
>>> So, I will go through the bisect session anyway, as it seems the best
>>> possibility to get a grip on it. But I don't want to make these builds
>>> on Windows. I'll build them on Linux and will try to ease my life a bit
>>> with some scripting.
>>>
>>> cu
>>> Jens
>>>
>>>
>>> Am 27.10.2023 um 17:11 schrieb Richard Zowalla:
>>>> Great news! (though I am not a Windows user anymore).
>>>>
>>>> You should be able to build TomEE on Windows, too. If that does not
>>>> work out-of-the-box, you could try to build within a docker container
>>>> or via git-bash in Windows
>>>>
>>>> (I am not 100% sure a Windows build will work out of the box)
>>>>
>>>> I agree, that it might complicate things for testing purpose as it
>>>> increases the amount of cycles needed, but that are already very good
>>>> insights!
>>>>
>>>> Gruß
>>>> Richard
>>>>
>>>>
>>>> Am Freitag, dem 27.10.2023 um 16:55 +0200 schrieb Jens Zurawski:
>>>>> I have a first result on this issue:
>>>>>
>>>>> The problem does not occur if TomEE is running on Linux. I've tested
>>>>> my
>>>>> own built 8.0.8, a freshly extracted 8.0.8 and 8.0.15 from the
>>>>> releases,
>>>>> and none of them showed the dev mode performance problem.
>>>>>
>>>>> Using TomEE 8.0.8 on Windows does show the dev mode performance
>>>>> problem.
>>>>> All of the tested ones, means the released 8.0.8 and my own built
>>>>> 8.0.8
>>>>> (the one I've built on Linux).
>>>>>
>>>>> My development environment is Windows 10 (Netbeans and therefore also
>>>>> the TomEE for easy deployment and debugging). The problem does occur
>>>>> on
>>>>> stand alone started TomEE and also if started out of Netbeans, so
>>>>> Netbeans can be sorted out as the root of the problem. Java Versions
>>>>> 11,
>>>>> 13, 15 and 17 tested, all the same, so Java should also not be the
>>>>> problem. BTW: It is very unlikely a problem of my individual Windows
>>>>> installation, because this problem also shows up on the Windows
>>>>> system
>>>>> of my colleague, which btw. was the root cause why I'm started to dig
>>>>> deeper into this problem now and why I subscribed to this mailing
>>>>> list ;-)
>>>>>
>>>>> I'm afraid, this will complicate my git bisect session a little bit
>>>>> :-(
>>>>>
>>>>> However, I'll keep you posted.
>>>>>
>>>>> cu
>>>>> Jens
>>>>>
>>>>>
>>>>> Am 27.10.2023 um 13:12 schrieb Jens Zurawski:
>>>>>> Thanks Richard (and Vicente),
>>>>>>
>>>>>> I've changed to JDK 8 and now the builds are successful (both 8.0.6
>>>>>> and 8.0.8).
>>>>>>
>>>>>> I'll script me some test scripts to ease the build, run and test
>>>>>> for
>>>>>> every bisect step.
>>>>>> A build needs around 3 to 4 minutes. Running a test with my
>>>>>> application will need some manual interaction and checks, I guess
>>>>>> also
>>>>>> something around 4 minutes. This sums up to roughly 8 minutes per
>>>>>> step.
>>>>>>
>>>>>> I've started a bisect and it told me:
>>>>>> git bisect good 20441eb
>>>>>> git bisect bad 4c8a616
>>>>>> Bisecting: 136 revisions left to test after this (roughly 7 steps)
>>>>>>
>>>>>> That means, if everything works fine, I should be able to identify
>>>>>> the
>>>>>> commit after about an hour? (That's cool.... why nobody has told me
>>>>>> about git bisect before? ;-) )
>>>>>>
>>>>>> Anyway, I will have to find some time for this, maybe I need a
>>>>>> couple
>>>>>> of days to find some.
>>>>>> I will let you know about the outcome.
>>>>>>
>>>>>> cu
>>>>>> Jens
>>>>>>
>>>>>>
>>>>>> Am 27.10.2023 um 11:50 schrieb Richard Zowalla:
>>>>>>> Hi,
>>>>>>>
>>>>>>> you need Java/JDK 8 to build TomEE 8.
>>>>>>>
>>>>>>> The Maven command is
>>>>>>>
>>>>>>> mvn --fail-at-end clean install -pl tomee/apache-tomee -am -
>>>>>>> Dfile.encoding=UTF-8 -DskipTests=true
>>>>>>>
>>>>>>> You will find the tar.gz/zip in ./tomee/apache-tomee/target
>>>>>>>
>>>>>>> Gruß
>>>>>>> Richard
>>>>>>>
>>>>>>> P.S. 8.0.7 is here:
>>>>>>> https://archive.apache.org/dist/tomee/tomee-8.0.7/
>>>>>>>
>>>>>>> Am Freitag, dem 27.10.2023 um 10:50 +0200 schrieb Jens Zurawski:
>>>>>>>> Hi Richard,
>>>>>>>>
>>>>>>>> to identify the "bad" version was easy. I didn't find a 8.0.7
>>>>>>>> release
>>>>>>>> on
>>>>>>>> the download page, so I started with 8.0.8. And this version
>>>>>>>> already
>>>>>>>> has
>>>>>>>> the slow dev mode. So for my problem I can say:
>>>>>>>> good: 8.0.6
>>>>>>>> bad: 8.0.8
>>>>>>>>
>>>>>>>> Now, I think, I need a little help from you. I tried to build a
>>>>>>>> TomEE
>>>>>>>> out of git (I've cloned the github repository
>>>>>>>> https://github.com/apache/tomee.git). I've tried these two
>>>>>>>> commits:
>>>>>>>> 20441eb (tag: tomee-8.0.6)
>>>>>>>> 4c8a616 (tag: tomee-project-8.0.8)
>>>>>>>>
>>>>>>>> But the build fails every time on "TomEE :: Container :: Core"
>>>>>>>> with:
>>>>>>>>
>>>>>>>> [INFO] BUILD FAILURE
>>>>>>>> [INFO]
>>>>>>>> ---------------------------------------------------------------
>>>>>>>> ------
>>>>>>>> ---
>>>>>>>> [INFO] Total time: 01:19 min
>>>>>>>> [INFO] Finished at: 2023-10-27T10:29:40+02:00
>>>>>>>> [INFO]
>>>>>>>> ---------------------------------------------------------------
>>>>>>>> ------
>>>>>>>> ---
>>>>>>>> [ERROR] Failed to execute goal
>>>>>>>> org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile
>>>>>>>> (default-compile) on project openejb-core: Compilation failure:
>>>>>>>> Compilation failure:
>>>>>>>> [ERROR]
>>>>>>>> /sandbox/tomee/container/openejb-
>>>>>>>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[21,
>>>>>>>> 21]
>>>>>>>> package org.omg.CORBA does not exist
>>>>>>>> [ERROR]
>>>>>>>> /sandbox/tomee/container/openejb-
>>>>>>>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[24,
>>>>>>>> 12]
>>>>>>>> cannot find symbol
>>>>>>>> [ERROR] symbol: class ORB
>>>>>>>> [ERROR] location: class org.apache.openejb.core.OrbFactory
>>>>>>>>
>>>>>>>> I've tried to build with:
>>>>>>>> mvn clean install
>>>>>>>>
>>>>>>>> My build environment is a linux ubuntu 20.04.6 LTS VM with:
>>>>>>>> git 2.25.1
>>>>>>>> Maven 3.6.3
>>>>>>>> java 14.0.2
>>>>>>>>
>>>>>>>> Is it some easy thing to fix, or do I have to switch some
>>>>>>>> versions of
>>>>>>>> my
>>>>>>>> environment?
>>>>>>>>
>>>>>>>> cu
>>>>>>>> Jens
>>>>>>>>
>>>>>>>> Am 26.10.2023 um 19:48 schrieb Richard Zowalla:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> without knowing code or having a reproducer, it might be not
>>>>>>>>> that
>>>>>>>>> obvious to find a reason / answer for the "academic problem"
>>>>>>>>> ;-)
>>>>>>>>>
>>>>>>>>> If you really want to boil it down, I recommend the following
>>>>>>>>> approach:
>>>>>>>>>
>>>>>>>>> (1) From your description it seems, that 8.0.6 is the last
>>>>>>>>> known
>>>>>>>>> "good"
>>>>>>>>> version.
>>>>>>>>>
>>>>>>>>> (2) You've investigated versions starting from 8.0.12
>>>>>>>>> onwards,
>>>>>>>>> which
>>>>>>>>> seems to be "bad".
>>>>>>>>>
>>>>>>>>> Following the approach recommended by Jonathan:
>>>>>>>>>
>>>>>>>>> - Can you check if it happens in 8.0.7 (8.0.8, ...) until we
>>>>>>>>> find
>>>>>>>>> the
>>>>>>>>> first "bad" released version?
>>>>>>>>>
>>>>>>>>> - If we have that, you could start a "git bisect" session
>>>>>>>>> between
>>>>>>>>> 8.0.6
>>>>>>>>> and the first version, which is known to be bad (reducing the
>>>>>>>>> amount of
>>>>>>>>> needed bisect cycles dramatically), so we find a commit /
>>>>>>>>> reason
>>>>>>>>> (if it
>>>>>>>>> is in TomEE).
>>>>>>>>>
>>>>>>>>> - It would involve some quick builds without executing tests
>>>>>>>>> (~10-
>>>>>>>>> 15min
>>>>>>>>> each) for each bisect step and testing on your side.
>>>>>>>>>
>>>>>>>>> Don't know if your inner monk is willing to go that route,
>>>>>>>>> but I am
>>>>>>>>> happy to help with the related Maven commands for quick
>>>>>>>>> building :)
>>>>>>>>>
>>>>>>>>> Gruß
>>>>>>>>> Richard
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Am Donnerstag, dem 26.10.2023 um 18:00 +0200 schrieb Jens
>>>>>>>>> Zurawski:
>>>>>>>>>> Hi Vicente,
>>>>>>>>>>
>>>>>>>>>> thank you for the tip. This is a good workaround. When I
>>>>>>>>>> switch
>>>>>>>>>> to
>>>>>>>>>> Production mode and insert this configuration, it reloads
>>>>>>>>>> the
>>>>>>>>>> view
>>>>>>>>>> from
>>>>>>>>>> the file if it has changed. This means, I can use a decent
>>>>>>>>>> version of
>>>>>>>>>> TomEE in my dev environment.
>>>>>>>>>>
>>>>>>>>>> However, it doesn't completely solve my problem, as the
>>>>>>>>>> error
>>>>>>>>>> reporting
>>>>>>>>>> isn't as verbose as in Dev mode as far as I can see. But I
>>>>>>>>>> have
>>>>>>>>>> to
>>>>>>>>>> check, maybe it's enough for my development.
>>>>>>>>>>
>>>>>>>>>> And, well.... the academic problem: "Why is it still broken
>>>>>>>>>> in
>>>>>>>>>> Dev
>>>>>>>>>> mode?" still runs circles through my brain ;-) Because if
>>>>>>>>>> it's a
>>>>>>>>>> problem
>>>>>>>>>> of my coding, it might be a good idea to change this,
>>>>>>>>>> before it
>>>>>>>>>> might
>>>>>>>>>> get a problem on Production mode, too, at some time in the
>>>>>>>>>> future.
>>>>>>>>>>
>>>>>>>>>> But anyway, you've helped me a good step forward, thanks.
>>>>>>>>>> cu
>>>>>>>>>> Jens
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Am 26.10.2023 um 17:04 schrieb Vicente Rossello:
>>>>>>>>>>> We are using the production stage with
>>>>>>>>>>> javax.faces.FACELETS_REFRESH_PERIOD
>>>>>>>>>>> = 1. It's faster and more similar to production, just try
>>>>>>>>>>> it if
>>>>>>>>>>> it
>>>>>>>>>>> solves
>>>>>>>>>>> your problem
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Oct 26, 2023 at 4:18 PM Jens Zurawski
>>>>>>>>>>> <jz...@diegurus.de>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> The one bundled with TomEE.
>>>>>>>>>>>> myfaces 2.3.7 on TomEE 8.0.6
>>>>>>>>>>>> and e.g. myfaces 2.3.9 on TomEE 8.0.12
>>>>>>>>>>>>
>>>>>>>>>>>> Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
>>>>>>>>>>>>> Which JSF lib are you using?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens
>>>>>>>>>>>>> Zurawski
>>>>>>>>>>>>> <jzu@diegurus.de
>>>>>>>>>>>>> :
>>>>>>>>>>>>>> Thank you for your quick reply, Jonathan.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It is good to know, that normally the Dev mode
>>>>>>>>>>>>>> should
>>>>>>>>>>>>>> work as
>>>>>>>>>>>>>> fast as
>>>>>>>>>>>> before even with 8.0.15. So it obviously has something
>>>>>>>>>>>> to do
>>>>>>>>>>>> with
>>>>>>>>>>>> my
>>>>>>>>>>>> environment, configuration and/or code.
>>>>>>>>>>>>>> And also thank you for the reference to tomitribe.
>>>>>>>>>>>>>> Unfortunately I'm
>>>>>>>>>>>> the only developer on this project, and it will be
>>>>>>>>>>>> difficult
>>>>>>>>>>>> to
>>>>>>>>>>>> convince my
>>>>>>>>>>>> customer to an additional support subscription, because
>>>>>>>>>>>> he on
>>>>>>>>>>>> production
>>>>>>>>>>>> doesn't have issues ;-) But, ok, that's my problem, and
>>>>>>>>>>>> maybe
>>>>>>>>>>>> I'll consult
>>>>>>>>>>>> tomitribe in the future for other projects.
>>>>>>>>>>>>>> So, I'm afraid, for the near future I'm on my own
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> solve
>>>>>>>>>>>>>> this
>>>>>>>>>>>> problem. Therefore any hints to restrict the places
>>>>>>>>>>>> where I
>>>>>>>>>>>> should take a
>>>>>>>>>>>> look will be more than welcome.
>>>>>>>>>>>>>> I'm a senior programmer with 20 years of experience
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>> Java
>>>>>>>>>>>>>> programming
>>>>>>>>>>>> (SE and JSP). But I'm relative new to
>>>>>>>>>>>> JSF/JavaEE/JakartaEE
>>>>>>>>>>>> and
>>>>>>>>>>>> sometimes I
>>>>>>>>>>>> have a hard time to understand which component of the
>>>>>>>>>>>> whole
>>>>>>>>>>>> stack
>>>>>>>>>>>> is
>>>>>>>>>>>> responsible for what.
>>>>>>>>>>>>>> The project meanwhile has over a 1/4 million lines
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>> code
>>>>>>>>>>>>>> (most of
>>>>>>>>>>>> them in the back-end, but also the front-end is big
>>>>>>>>>>>> enough to
>>>>>>>>>>>> not
>>>>>>>>>>>> trace
>>>>>>>>>>>> through every lane if I want to get to the point in
>>>>>>>>>>>> this life
>>>>>>>>>>>> :-D
>>>>>>>>>>>> ).
>>>>>>>>>>>>>> The application now runs for nearly 2 years on the
>>>>>>>>>>>>>> customers
>>>>>>>>>>>>>> site and
>>>>>>>>>>>> is very reliable and fast in its daily usage. Apart
>>>>>>>>>>>> from
>>>>>>>>>>>> occasional updates
>>>>>>>>>>>> every few month, it will never be restarted or need any
>>>>>>>>>>>> manual
>>>>>>>>>>>> maintenance
>>>>>>>>>>>> at all. So I assume the code is not the badest on
>>>>>>>>>>>> earth. At
>>>>>>>>>>>> least
>>>>>>>>>>>> because
>>>>>>>>>>>> of that I'm able to out-rule things like memory leaks,
>>>>>>>>>>>> gc
>>>>>>>>>>>> issues
>>>>>>>>>>>> or
>>>>>>>>>>>> bottlenecks (max Heap is never reached, neither half of
>>>>>>>>>>>> it,
>>>>>>>>>>>> even
>>>>>>>>>>>> after
>>>>>>>>>>>> running for month).
>>>>>>>>>>>>>> Of course there could be aspects of JSF which I may
>>>>>>>>>>>>>> not
>>>>>>>>>>>>>> have
>>>>>>>>>>>>>> understood
>>>>>>>>>>>> correctly until now. But because there are so many new
>>>>>>>>>>>> things
>>>>>>>>>>>> to
>>>>>>>>>>>> a JSF
>>>>>>>>>>>> beginner like me, I'm running out of ideas what this
>>>>>>>>>>>> could
>>>>>>>>>>>> be. It
>>>>>>>>>>>> is
>>>>>>>>>>>> something that worked well 'til TomEE 8.0.6 (Prod and
>>>>>>>>>>>> Dev
>>>>>>>>>>>> mode)
>>>>>>>>>>>> and
>>>>>>>>>>>> suddenly don't work anymore on 8.0.12 in Dev mode (but
>>>>>>>>>>>> still
>>>>>>>>>>>> works great in
>>>>>>>>>>>> Prod mode). There are only minor versions in-between.
>>>>>>>>>>>> So I
>>>>>>>>>>>> think
>>>>>>>>>>>> there were
>>>>>>>>>>>> no revolutionary changes anywhere. My hope was (and
>>>>>>>>>>>> still
>>>>>>>>>>>> is),
>>>>>>>>>>>> that anybody
>>>>>>>>>>>> here on the list might have had a similar problem and
>>>>>>>>>>>> has
>>>>>>>>>>>> solved
>>>>>>>>>>>> it, or
>>>>>>>>>>>> maybe someone knows about some changes in Dev mode
>>>>>>>>>>>> which
>>>>>>>>>>>> could
>>>>>>>>>>>> explain such
>>>>>>>>>>>> behaviour (don't know, maybe xml parsing or component
>>>>>>>>>>>> tree
>>>>>>>>>>>> generation has
>>>>>>>>>>>> changed, or scope handling of managed beans, or
>>>>>>>>>>>> whatever
>>>>>>>>>>>> could
>>>>>>>>>>>> cause some
>>>>>>>>>>>> code to be entered far more often than before, or some
>>>>>>>>>>>> cache/pool
>>>>>>>>>>>> was
>>>>>>>>>>>> removed or has different behaviour... such things).
>>>>>>>>>>>> Then it
>>>>>>>>>>>> would
>>>>>>>>>>>> be far
>>>>>>>>>>>> more easy for me to track down the real problem.
>>>>>>>>>>>>>> Anyway, when I get some spare time (currently have
>>>>>>>>>>>>>> some
>>>>>>>>>>>>>> deadlines in
>>>>>>>>>>>> sight), I'll try to build a small test case. Either I
>>>>>>>>>>>> can get
>>>>>>>>>>>> a
>>>>>>>>>>>> grip on the
>>>>>>>>>>>> issue myself on doing this, or I can supply it to
>>>>>>>>>>>> anyone who
>>>>>>>>>>>> might want to
>>>>>>>>>>>> have a look at it.
>>>>>>>>>>>>>> cu
>>>>>>>>>>>>>> Jens
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
>>>>>>>>>>>>>>>> newer versions are so incredibly slow in dev
>>>>>>>>>>>>>>>> mode
>>>>>>>>>>>>>>> I can assure you that Dev/Prod mode works very
>>>>>>>>>>>>>>> very
>>>>>>>>>>>>>>> swiftly
>>>>>>>>>>>>>>> on 8.0.15
>>>>>>>>>>>>>>> for sizable applications. For example, I have a
>>>>>>>>>>>>>>> giant
>>>>>>>>>>>>>>> application
>>>>>>>>>>>>>>> (thousands of LOC, 50+ jsf views) that has sub-
>>>>>>>>>>>>>>> 8ms
>>>>>>>>>>>>>>> response
>>>>>>>>>>>>>>> times
>>>>>>>>>>>>>>> (minus database time). More than likely they are
>>>>>>>>>>>>>>> doing
>>>>>>>>>>>>>>> something they
>>>>>>>>>>>>>>> should not be doing with JSF apis and are causing
>>>>>>>>>>>>>>> problems.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I would check the obvious things first with
>>>>>>>>>>>>>>> VisualVM:
>>>>>>>>>>>>>>> heap
>>>>>>>>>>>>>>> size, gc
>>>>>>>>>>>>>>> frequency, etc. You can also run their profiler
>>>>>>>>>>>>>>> and see
>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>> you can
>>>>>>>>>>>>>>> spot anything immediately obvious.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> After that, the easiest way to locate the code
>>>>>>>>>>>>>>> slowing
>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>> down is
>>>>>>>>>>>>>>> bisection. Cut half the code out, run, and
>>>>>>>>>>>>>>> continue
>>>>>>>>>>>>>>> cutting
>>>>>>>>>>>>>>> until you
>>>>>>>>>>>>>>> can locate the problematic code. Then after that,
>>>>>>>>>>>>>>> isolate a
>>>>>>>>>>>>>>> test case
>>>>>>>>>>>>>>> where flipping a boolean flag triggers the issue
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> post
>>>>>>>>>>>>>>> the results
>>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I would get ahold of:
>>>>>>>>>>>>>>> https://www.tomitribe.com who are
>>>>>>>>>>>>>>> literal
>>>>>>>>>>>>>>> experts in this stuff. They offer professional
>>>>>>>>>>>>>>> services
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> handle
>>>>>>>>>>>>>>> exactly these situations and can work directly in
>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>> codebase to
>>>>>>>>>>>>>>> help you find and fix the problem.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski
>>>>>>>>>>>>>>> <jz...@diegurus.de> wrote:
>>>>>>>>>>>>>>>> Hi altogether,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm new to this list and hope it's the right
>>>>>>>>>>>>>>>> place to
>>>>>>>>>>>>>>>> ask
>>>>>>>>>>>>>>>> this
>>>>>>>>>>>> question.
>>>>>>>>>>>>>>>> If not, and someone know the right place to ask
>>>>>>>>>>>>>>>> this,
>>>>>>>>>>>>>>>> please give
>>>>>>>>>>>> advise.
>>>>>>>>>>>>>>>> I'm developing a big JSF Application for a
>>>>>>>>>>>>>>>> customer.
>>>>>>>>>>>>>>>> It's
>>>>>>>>>>>>>>>> a long
>>>>>>>>>>>> running
>>>>>>>>>>>>>>>> project and development and when I started with
>>>>>>>>>>>>>>>> it,
>>>>>>>>>>>>>>>> TomEE
>>>>>>>>>>>>>>>> 8.0.6 was
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> most recent Version of TomEE. The switch to
>>>>>>>>>>>>>>>> Jakarta
>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>> planned for
>>>>>>>>>>>> next
>>>>>>>>>>>>>>>> year, so I'm still on the 8.x path for the time
>>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> wasn't able
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> test if this problem still exists in the 9.x
>>>>>>>>>>>>>>>> branch.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The problem:
>>>>>>>>>>>>>>>> On the customers site I'm with the latest TomEE
>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>> 8.0.15
>>>>>>>>>>>> (running
>>>>>>>>>>>>>>>> on Java 11) and everything works fine, because
>>>>>>>>>>>>>>>> it is
>>>>>>>>>>>>>>>> running in
>>>>>>>>>>>>>>>> production mode. But in my development
>>>>>>>>>>>>>>>> environment
>>>>>>>>>>>>>>>> I'm
>>>>>>>>>>>>>>>> still stuck
>>>>>>>>>>>> with
>>>>>>>>>>>>>>>> 8.0.6, because there I need the development
>>>>>>>>>>>>>>>> mode
>>>>>>>>>>>>>>>> (javax.faces.PROJECT_STAGE: Development). I
>>>>>>>>>>>>>>>> need to
>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>> able to see
>>>>>>>>>>>>>>>> changes in facelets without restarting
>>>>>>>>>>>>>>>> everything
>>>>>>>>>>>>>>>> everytime, and to
>>>>>>>>>>>> get
>>>>>>>>>>>>>>>> more detailed error messages.
>>>>>>>>>>>>>>>> My attempts to update my dev TomEE to something
>>>>>>>>>>>>>>>> newer
>>>>>>>>>>>>>>>> than 8.0.6 all
>>>>>>>>>>>>>>>> failed, because all (at least all I've tested
>>>>>>>>>>>>>>>> so far)
>>>>>>>>>>>>>>>> newer versions
>>>>>>>>>>>> are
>>>>>>>>>>>>>>>> so incredibly slow in dev mode, that it's
>>>>>>>>>>>>>>>> unbearable
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> use the
>>>>>>>>>>>>>>>> application. Several very looong seconds on
>>>>>>>>>>>>>>>> every
>>>>>>>>>>>>>>>> request
>>>>>>>>>>>>>>>> (even the
>>>>>>>>>>>>>>>> little AJAX requests in a view) is simply not a
>>>>>>>>>>>>>>>> practical
>>>>>>>>>>>>>>>> environment.
>>>>>>>>>>>>>>>> After switching to Production mode everything
>>>>>>>>>>>>>>>> works
>>>>>>>>>>>>>>>> very
>>>>>>>>>>>>>>>> fast, and all
>>>>>>>>>>>>>>>> my views have response times of very few ms
>>>>>>>>>>>>>>>> (even the
>>>>>>>>>>>>>>>> big
>>>>>>>>>>>>>>>> ones with
>>>>>>>>>>>> max.
>>>>>>>>>>>>>>>> around 400ms). When switching back to
>>>>>>>>>>>>>>>> Development
>>>>>>>>>>>>>>>> mode I
>>>>>>>>>>>>>>>> have response
>>>>>>>>>>>>>>>> times of up to 30s on big views. Not with
>>>>>>>>>>>>>>>> 8.0.6,
>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>> even in
>>>>>>>>>>>>>>>> development mode it's reasonable fast in not
>>>>>>>>>>>>>>>> getting
>>>>>>>>>>>>>>>> higher than 1s.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> My question:
>>>>>>>>>>>>>>>> What causes this enormous performance
>>>>>>>>>>>>>>>> degradation?
>>>>>>>>>>>>>>>> I'm
>>>>>>>>>>>>>>>> hoping, it is
>>>>>>>>>>>>>>>> just a configuration which now has another
>>>>>>>>>>>>>>>> default
>>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>>> or the like.
>>>>>>>>>>>> If
>>>>>>>>>>>>>>>> yes, maybe someone can point me in the right
>>>>>>>>>>>>>>>> direction of
>>>>>>>>>>>>>>>> where to
>>>>>>>>>>>> find
>>>>>>>>>>>>>>>> this configuration? If it's not a configuration
>>>>>>>>>>>>>>>> thing:
>>>>>>>>>>>>>>>> what can I do
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> get around this?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I haven't tested all Versions of TomEE from
>>>>>>>>>>>>>>>> 8.0.6 to
>>>>>>>>>>>>>>>> 8.0.15, so I
>>>>>>>>>>>> can't
>>>>>>>>>>>>>>>> say at what version exactly this behaviour
>>>>>>>>>>>>>>>> changes.
>>>>>>>>>>>>>>>> If it
>>>>>>>>>>>>>>>> helps or is
>>>>>>>>>>>>>>>> needed, I can make some tests to find the exact
>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>> where this
>>>>>>>>>>>>>>>> happens. Versions I've already tested are:
>>>>>>>>>>>>>>>> 8.0.12,
>>>>>>>>>>>>>>>> 8.0.13
>>>>>>>>>>>>>>>> and 8.0.15.
>>>>>>>>>>>>>>>> All of them are very slow in dev mode. If you
>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>> information,
>>>>>>>>>>>>>>>> I'll try to provide it.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks in advance for any help
>>>>>>>>>>>>>>>> cu
>>>>>>>>>>>>>>>> Jens
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Vicente Rossello <co...@gmail.com>.
Sorry, that was not correct.
The allowLinking is at the resources level. So, in the META-INF/context.xml
file:
<?xml version="1.0" encoding="UTF-8"?>
<Context allowCasualMultipartParsing="true" reloadable="false"
containerSciFilter="org.apache.tomcat.websocket.server.WsSci|org.apache.jasper.servlet.JasperInitializer"
parallelAnnotationScanning="true"
clearReferencesThreadLocals="false"
skipMemoryLeakChecksOnJvmShutdown="true">
<Manager pathname=""/>
<Resources archiveIndexStrategy="bloom" allowLinking="true"/>
</Context>
BTW, the slowness is something between checking the canonicalPath too
many times and jdk12 that removed those canonCache by default. See
https://bugs.openjdk.org/browse/JDK-8258036
On Sat, Oct 28, 2023 at 8:07 AM Vicente Rossello <co...@gmail.com>
wrote:
> Hi,
>
> It is a tomcat problem. The real problem is in DirResourcesSet. It is
> calling a lot of times File.getCanonicalPath(). This method is very slow in
> windows.
>
> What I did is extending the DirResourcesSet to skip processing the
> getCanonicalPath method so many times, but I really didn't understand the
> problem.
>
> A simpler and better solution is to set allowLinking="true" in the
> context.xml file. So tomcat does not validate the canonicalPath every
> single time, which is just too much.
>
> Best regards,
> Vicente
>
> On Fri, Oct 27, 2023 at 5:51 PM Jens Zurawski <jz...@diegurus.de> wrote:
>
>> My client is the last Windows System in my little universe. Due to some
>> constraints with my customers I have to deal with it for the time being.
>>
>> The switches, Vicente has provided, solved the problem for me so far.
>> But my "inner monk" is still not satisfied, because practical no one
>> knows of these switches, and after all, TomEE should do a good job on
>> Windows, too. And because 8.0.6 does a good job, and 8.0.8 breaks it to
>> some degree, we should at least try to figure out, what exactly causes
>> this break. Maybe there is an easy way to fix it, then it should be
>> fixed IMHO.
>>
>> So, I will go through the bisect session anyway, as it seems the best
>> possibility to get a grip on it. But I don't want to make these builds
>> on Windows. I'll build them on Linux and will try to ease my life a bit
>> with some scripting.
>>
>> cu
>> Jens
>>
>>
>> Am 27.10.2023 um 17:11 schrieb Richard Zowalla:
>> > Great news! (though I am not a Windows user anymore).
>> >
>> > You should be able to build TomEE on Windows, too. If that does not
>> > work out-of-the-box, you could try to build within a docker container
>> > or via git-bash in Windows
>> >
>> > (I am not 100% sure a Windows build will work out of the box)
>> >
>> > I agree, that it might complicate things for testing purpose as it
>> > increases the amount of cycles needed, but that are already very good
>> > insights!
>> >
>> > Gruß
>> > Richard
>> >
>> >
>> > Am Freitag, dem 27.10.2023 um 16:55 +0200 schrieb Jens Zurawski:
>> >> I have a first result on this issue:
>> >>
>> >> The problem does not occur if TomEE is running on Linux. I've tested
>> >> my
>> >> own built 8.0.8, a freshly extracted 8.0.8 and 8.0.15 from the
>> >> releases,
>> >> and none of them showed the dev mode performance problem.
>> >>
>> >> Using TomEE 8.0.8 on Windows does show the dev mode performance
>> >> problem.
>> >> All of the tested ones, means the released 8.0.8 and my own built
>> >> 8.0.8
>> >> (the one I've built on Linux).
>> >>
>> >> My development environment is Windows 10 (Netbeans and therefore also
>> >> the TomEE for easy deployment and debugging). The problem does occur
>> >> on
>> >> stand alone started TomEE and also if started out of Netbeans, so
>> >> Netbeans can be sorted out as the root of the problem. Java Versions
>> >> 11,
>> >> 13, 15 and 17 tested, all the same, so Java should also not be the
>> >> problem. BTW: It is very unlikely a problem of my individual Windows
>> >> installation, because this problem also shows up on the Windows
>> >> system
>> >> of my colleague, which btw. was the root cause why I'm started to dig
>> >> deeper into this problem now and why I subscribed to this mailing
>> >> list ;-)
>> >>
>> >> I'm afraid, this will complicate my git bisect session a little bit
>> >> :-(
>> >>
>> >> However, I'll keep you posted.
>> >>
>> >> cu
>> >> Jens
>> >>
>> >>
>> >> Am 27.10.2023 um 13:12 schrieb Jens Zurawski:
>> >>> Thanks Richard (and Vicente),
>> >>>
>> >>> I've changed to JDK 8 and now the builds are successful (both 8.0.6
>> >>> and 8.0.8).
>> >>>
>> >>> I'll script me some test scripts to ease the build, run and test
>> >>> for
>> >>> every bisect step.
>> >>> A build needs around 3 to 4 minutes. Running a test with my
>> >>> application will need some manual interaction and checks, I guess
>> >>> also
>> >>> something around 4 minutes. This sums up to roughly 8 minutes per
>> >>> step.
>> >>>
>> >>> I've started a bisect and it told me:
>> >>> git bisect good 20441eb
>> >>> git bisect bad 4c8a616
>> >>> Bisecting: 136 revisions left to test after this (roughly 7 steps)
>> >>>
>> >>> That means, if everything works fine, I should be able to identify
>> >>> the
>> >>> commit after about an hour? (That's cool.... why nobody has told me
>> >>> about git bisect before? ;-) )
>> >>>
>> >>> Anyway, I will have to find some time for this, maybe I need a
>> >>> couple
>> >>> of days to find some.
>> >>> I will let you know about the outcome.
>> >>>
>> >>> cu
>> >>> Jens
>> >>>
>> >>>
>> >>> Am 27.10.2023 um 11:50 schrieb Richard Zowalla:
>> >>>> Hi,
>> >>>>
>> >>>> you need Java/JDK 8 to build TomEE 8.
>> >>>>
>> >>>> The Maven command is
>> >>>>
>> >>>> mvn --fail-at-end clean install -pl tomee/apache-tomee -am -
>> >>>> Dfile.encoding=UTF-8 -DskipTests=true
>> >>>>
>> >>>> You will find the tar.gz/zip in ./tomee/apache-tomee/target
>> >>>>
>> >>>> Gruß
>> >>>> Richard
>> >>>>
>> >>>> P.S. 8.0.7 is here:
>> >>>> https://archive.apache.org/dist/tomee/tomee-8.0.7/
>> >>>>
>> >>>> Am Freitag, dem 27.10.2023 um 10:50 +0200 schrieb Jens Zurawski:
>> >>>>> Hi Richard,
>> >>>>>
>> >>>>> to identify the "bad" version was easy. I didn't find a 8.0.7
>> >>>>> release
>> >>>>> on
>> >>>>> the download page, so I started with 8.0.8. And this version
>> >>>>> already
>> >>>>> has
>> >>>>> the slow dev mode. So for my problem I can say:
>> >>>>> good: 8.0.6
>> >>>>> bad: 8.0.8
>> >>>>>
>> >>>>> Now, I think, I need a little help from you. I tried to build a
>> >>>>> TomEE
>> >>>>> out of git (I've cloned the github repository
>> >>>>> https://github.com/apache/tomee.git). I've tried these two
>> >>>>> commits:
>> >>>>> 20441eb (tag: tomee-8.0.6)
>> >>>>> 4c8a616 (tag: tomee-project-8.0.8)
>> >>>>>
>> >>>>> But the build fails every time on "TomEE :: Container :: Core"
>> >>>>> with:
>> >>>>>
>> >>>>> [INFO] BUILD FAILURE
>> >>>>> [INFO]
>> >>>>> ---------------------------------------------------------------
>> >>>>> ------
>> >>>>> ---
>> >>>>> [INFO] Total time: 01:19 min
>> >>>>> [INFO] Finished at: 2023-10-27T10:29:40+02:00
>> >>>>> [INFO]
>> >>>>> ---------------------------------------------------------------
>> >>>>> ------
>> >>>>> ---
>> >>>>> [ERROR] Failed to execute goal
>> >>>>> org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile
>> >>>>> (default-compile) on project openejb-core: Compilation failure:
>> >>>>> Compilation failure:
>> >>>>> [ERROR]
>> >>>>> /sandbox/tomee/container/openejb-
>> >>>>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[21,
>> >>>>> 21]
>> >>>>> package org.omg.CORBA does not exist
>> >>>>> [ERROR]
>> >>>>> /sandbox/tomee/container/openejb-
>> >>>>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[24,
>> >>>>> 12]
>> >>>>> cannot find symbol
>> >>>>> [ERROR] symbol: class ORB
>> >>>>> [ERROR] location: class org.apache.openejb.core.OrbFactory
>> >>>>>
>> >>>>> I've tried to build with:
>> >>>>> mvn clean install
>> >>>>>
>> >>>>> My build environment is a linux ubuntu 20.04.6 LTS VM with:
>> >>>>> git 2.25.1
>> >>>>> Maven 3.6.3
>> >>>>> java 14.0.2
>> >>>>>
>> >>>>> Is it some easy thing to fix, or do I have to switch some
>> >>>>> versions of
>> >>>>> my
>> >>>>> environment?
>> >>>>>
>> >>>>> cu
>> >>>>> Jens
>> >>>>>
>> >>>>> Am 26.10.2023 um 19:48 schrieb Richard Zowalla:
>> >>>>>> Hi,
>> >>>>>>
>> >>>>>> without knowing code or having a reproducer, it might be not
>> >>>>>> that
>> >>>>>> obvious to find a reason / answer for the "academic problem"
>> >>>>>> ;-)
>> >>>>>>
>> >>>>>> If you really want to boil it down, I recommend the following
>> >>>>>> approach:
>> >>>>>>
>> >>>>>> (1) From your description it seems, that 8.0.6 is the last
>> >>>>>> known
>> >>>>>> "good"
>> >>>>>> version.
>> >>>>>>
>> >>>>>> (2) You've investigated versions starting from 8.0.12
>> >>>>>> onwards,
>> >>>>>> which
>> >>>>>> seems to be "bad".
>> >>>>>>
>> >>>>>> Following the approach recommended by Jonathan:
>> >>>>>>
>> >>>>>> - Can you check if it happens in 8.0.7 (8.0.8, ...) until we
>> >>>>>> find
>> >>>>>> the
>> >>>>>> first "bad" released version?
>> >>>>>>
>> >>>>>> - If we have that, you could start a "git bisect" session
>> >>>>>> between
>> >>>>>> 8.0.6
>> >>>>>> and the first version, which is known to be bad (reducing the
>> >>>>>> amount of
>> >>>>>> needed bisect cycles dramatically), so we find a commit /
>> >>>>>> reason
>> >>>>>> (if it
>> >>>>>> is in TomEE).
>> >>>>>>
>> >>>>>> - It would involve some quick builds without executing tests
>> >>>>>> (~10-
>> >>>>>> 15min
>> >>>>>> each) for each bisect step and testing on your side.
>> >>>>>>
>> >>>>>> Don't know if your inner monk is willing to go that route,
>> >>>>>> but I am
>> >>>>>> happy to help with the related Maven commands for quick
>> >>>>>> building :)
>> >>>>>>
>> >>>>>> Gruß
>> >>>>>> Richard
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> Am Donnerstag, dem 26.10.2023 um 18:00 +0200 schrieb Jens
>> >>>>>> Zurawski:
>> >>>>>>> Hi Vicente,
>> >>>>>>>
>> >>>>>>> thank you for the tip. This is a good workaround. When I
>> >>>>>>> switch
>> >>>>>>> to
>> >>>>>>> Production mode and insert this configuration, it reloads
>> >>>>>>> the
>> >>>>>>> view
>> >>>>>>> from
>> >>>>>>> the file if it has changed. This means, I can use a decent
>> >>>>>>> version of
>> >>>>>>> TomEE in my dev environment.
>> >>>>>>>
>> >>>>>>> However, it doesn't completely solve my problem, as the
>> >>>>>>> error
>> >>>>>>> reporting
>> >>>>>>> isn't as verbose as in Dev mode as far as I can see. But I
>> >>>>>>> have
>> >>>>>>> to
>> >>>>>>> check, maybe it's enough for my development.
>> >>>>>>>
>> >>>>>>> And, well.... the academic problem: "Why is it still broken
>> >>>>>>> in
>> >>>>>>> Dev
>> >>>>>>> mode?" still runs circles through my brain ;-) Because if
>> >>>>>>> it's a
>> >>>>>>> problem
>> >>>>>>> of my coding, it might be a good idea to change this,
>> >>>>>>> before it
>> >>>>>>> might
>> >>>>>>> get a problem on Production mode, too, at some time in the
>> >>>>>>> future.
>> >>>>>>>
>> >>>>>>> But anyway, you've helped me a good step forward, thanks.
>> >>>>>>> cu
>> >>>>>>> Jens
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Am 26.10.2023 um 17:04 schrieb Vicente Rossello:
>> >>>>>>>> We are using the production stage with
>> >>>>>>>> javax.faces.FACELETS_REFRESH_PERIOD
>> >>>>>>>> = 1. It's faster and more similar to production, just try
>> >>>>>>>> it if
>> >>>>>>>> it
>> >>>>>>>> solves
>> >>>>>>>> your problem
>> >>>>>>>>
>> >>>>>>>> On Thu, Oct 26, 2023 at 4:18 PM Jens Zurawski
>> >>>>>>>> <jz...@diegurus.de>
>> >>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> The one bundled with TomEE.
>> >>>>>>>>> myfaces 2.3.7 on TomEE 8.0.6
>> >>>>>>>>> and e.g. myfaces 2.3.9 on TomEE 8.0.12
>> >>>>>>>>>
>> >>>>>>>>> Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
>> >>>>>>>>>> Which JSF lib are you using?
>> >>>>>>>>>>
>> >>>>>>>>>> Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens
>> >>>>>>>>>> Zurawski
>> >>>>>>>>>> <jzu@diegurus.de
>> >>>>>>>>>> :
>> >>>>>>>>>>> Thank you for your quick reply, Jonathan.
>> >>>>>>>>>>>
>> >>>>>>>>>>> It is good to know, that normally the Dev mode
>> >>>>>>>>>>> should
>> >>>>>>>>>>> work as
>> >>>>>>>>>>> fast as
>> >>>>>>>>> before even with 8.0.15. So it obviously has something
>> >>>>>>>>> to do
>> >>>>>>>>> with
>> >>>>>>>>> my
>> >>>>>>>>> environment, configuration and/or code.
>> >>>>>>>>>>> And also thank you for the reference to tomitribe.
>> >>>>>>>>>>> Unfortunately I'm
>> >>>>>>>>> the only developer on this project, and it will be
>> >>>>>>>>> difficult
>> >>>>>>>>> to
>> >>>>>>>>> convince my
>> >>>>>>>>> customer to an additional support subscription, because
>> >>>>>>>>> he on
>> >>>>>>>>> production
>> >>>>>>>>> doesn't have issues ;-) But, ok, that's my problem, and
>> >>>>>>>>> maybe
>> >>>>>>>>> I'll consult
>> >>>>>>>>> tomitribe in the future for other projects.
>> >>>>>>>>>>> So, I'm afraid, for the near future I'm on my own
>> >>>>>>>>>>> to
>> >>>>>>>>>>> solve
>> >>>>>>>>>>> this
>> >>>>>>>>> problem. Therefore any hints to restrict the places
>> >>>>>>>>> where I
>> >>>>>>>>> should take a
>> >>>>>>>>> look will be more than welcome.
>> >>>>>>>>>>> I'm a senior programmer with 20 years of experience
>> >>>>>>>>>>> in
>> >>>>>>>>>>> Java
>> >>>>>>>>>>> programming
>> >>>>>>>>> (SE and JSP). But I'm relative new to
>> >>>>>>>>> JSF/JavaEE/JakartaEE
>> >>>>>>>>> and
>> >>>>>>>>> sometimes I
>> >>>>>>>>> have a hard time to understand which component of the
>> >>>>>>>>> whole
>> >>>>>>>>> stack
>> >>>>>>>>> is
>> >>>>>>>>> responsible for what.
>> >>>>>>>>>>> The project meanwhile has over a 1/4 million lines
>> >>>>>>>>>>> of
>> >>>>>>>>>>> code
>> >>>>>>>>>>> (most of
>> >>>>>>>>> them in the back-end, but also the front-end is big
>> >>>>>>>>> enough to
>> >>>>>>>>> not
>> >>>>>>>>> trace
>> >>>>>>>>> through every lane if I want to get to the point in
>> >>>>>>>>> this life
>> >>>>>>>>> :-D
>> >>>>>>>>> ).
>> >>>>>>>>>>> The application now runs for nearly 2 years on the
>> >>>>>>>>>>> customers
>> >>>>>>>>>>> site and
>> >>>>>>>>> is very reliable and fast in its daily usage. Apart
>> >>>>>>>>> from
>> >>>>>>>>> occasional updates
>> >>>>>>>>> every few month, it will never be restarted or need any
>> >>>>>>>>> manual
>> >>>>>>>>> maintenance
>> >>>>>>>>> at all. So I assume the code is not the badest on
>> >>>>>>>>> earth. At
>> >>>>>>>>> least
>> >>>>>>>>> because
>> >>>>>>>>> of that I'm able to out-rule things like memory leaks,
>> >>>>>>>>> gc
>> >>>>>>>>> issues
>> >>>>>>>>> or
>> >>>>>>>>> bottlenecks (max Heap is never reached, neither half of
>> >>>>>>>>> it,
>> >>>>>>>>> even
>> >>>>>>>>> after
>> >>>>>>>>> running for month).
>> >>>>>>>>>>> Of course there could be aspects of JSF which I may
>> >>>>>>>>>>> not
>> >>>>>>>>>>> have
>> >>>>>>>>>>> understood
>> >>>>>>>>> correctly until now. But because there are so many new
>> >>>>>>>>> things
>> >>>>>>>>> to
>> >>>>>>>>> a JSF
>> >>>>>>>>> beginner like me, I'm running out of ideas what this
>> >>>>>>>>> could
>> >>>>>>>>> be. It
>> >>>>>>>>> is
>> >>>>>>>>> something that worked well 'til TomEE 8.0.6 (Prod and
>> >>>>>>>>> Dev
>> >>>>>>>>> mode)
>> >>>>>>>>> and
>> >>>>>>>>> suddenly don't work anymore on 8.0.12 in Dev mode (but
>> >>>>>>>>> still
>> >>>>>>>>> works great in
>> >>>>>>>>> Prod mode). There are only minor versions in-between.
>> >>>>>>>>> So I
>> >>>>>>>>> think
>> >>>>>>>>> there were
>> >>>>>>>>> no revolutionary changes anywhere. My hope was (and
>> >>>>>>>>> still
>> >>>>>>>>> is),
>> >>>>>>>>> that anybody
>> >>>>>>>>> here on the list might have had a similar problem and
>> >>>>>>>>> has
>> >>>>>>>>> solved
>> >>>>>>>>> it, or
>> >>>>>>>>> maybe someone knows about some changes in Dev mode
>> >>>>>>>>> which
>> >>>>>>>>> could
>> >>>>>>>>> explain such
>> >>>>>>>>> behaviour (don't know, maybe xml parsing or component
>> >>>>>>>>> tree
>> >>>>>>>>> generation has
>> >>>>>>>>> changed, or scope handling of managed beans, or
>> >>>>>>>>> whatever
>> >>>>>>>>> could
>> >>>>>>>>> cause some
>> >>>>>>>>> code to be entered far more often than before, or some
>> >>>>>>>>> cache/pool
>> >>>>>>>>> was
>> >>>>>>>>> removed or has different behaviour... such things).
>> >>>>>>>>> Then it
>> >>>>>>>>> would
>> >>>>>>>>> be far
>> >>>>>>>>> more easy for me to track down the real problem.
>> >>>>>>>>>>> Anyway, when I get some spare time (currently have
>> >>>>>>>>>>> some
>> >>>>>>>>>>> deadlines in
>> >>>>>>>>> sight), I'll try to build a small test case. Either I
>> >>>>>>>>> can get
>> >>>>>>>>> a
>> >>>>>>>>> grip on the
>> >>>>>>>>> issue myself on doing this, or I can supply it to
>> >>>>>>>>> anyone who
>> >>>>>>>>> might want to
>> >>>>>>>>> have a look at it.
>> >>>>>>>>>>> cu
>> >>>>>>>>>>> Jens
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
>> >>>>>>>>>>>>> newer versions are so incredibly slow in dev
>> >>>>>>>>>>>>> mode
>> >>>>>>>>>>>> I can assure you that Dev/Prod mode works very
>> >>>>>>>>>>>> very
>> >>>>>>>>>>>> swiftly
>> >>>>>>>>>>>> on 8.0.15
>> >>>>>>>>>>>> for sizable applications. For example, I have a
>> >>>>>>>>>>>> giant
>> >>>>>>>>>>>> application
>> >>>>>>>>>>>> (thousands of LOC, 50+ jsf views) that has sub-
>> >>>>>>>>>>>> 8ms
>> >>>>>>>>>>>> response
>> >>>>>>>>>>>> times
>> >>>>>>>>>>>> (minus database time). More than likely they are
>> >>>>>>>>>>>> doing
>> >>>>>>>>>>>> something they
>> >>>>>>>>>>>> should not be doing with JSF apis and are causing
>> >>>>>>>>>>>> problems.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> I would check the obvious things first with
>> >>>>>>>>>>>> VisualVM:
>> >>>>>>>>>>>> heap
>> >>>>>>>>>>>> size, gc
>> >>>>>>>>>>>> frequency, etc. You can also run their profiler
>> >>>>>>>>>>>> and see
>> >>>>>>>>>>>> if
>> >>>>>>>>>>>> you can
>> >>>>>>>>>>>> spot anything immediately obvious.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> After that, the easiest way to locate the code
>> >>>>>>>>>>>> slowing
>> >>>>>>>>>>>> you
>> >>>>>>>>>>>> down is
>> >>>>>>>>>>>> bisection. Cut half the code out, run, and
>> >>>>>>>>>>>> continue
>> >>>>>>>>>>>> cutting
>> >>>>>>>>>>>> until you
>> >>>>>>>>>>>> can locate the problematic code. Then after that,
>> >>>>>>>>>>>> isolate a
>> >>>>>>>>>>>> test case
>> >>>>>>>>>>>> where flipping a boolean flag triggers the issue
>> >>>>>>>>>>>> and
>> >>>>>>>>>>>> post
>> >>>>>>>>>>>> the results
>> >>>>>>>>>>>> here.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> I would get ahold of:
>> >>>>>>>>>>>> https://www.tomitribe.com who are
>> >>>>>>>>>>>> literal
>> >>>>>>>>>>>> experts in this stuff. They offer professional
>> >>>>>>>>>>>> services
>> >>>>>>>>>>>> to
>> >>>>>>>>>>>> handle
>> >>>>>>>>>>>> exactly these situations and can work directly in
>> >>>>>>>>>>>> your
>> >>>>>>>>>>>> codebase to
>> >>>>>>>>>>>> help you find and fix the problem.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski
>> >>>>>>>>>>>> <jz...@diegurus.de> wrote:
>> >>>>>>>>>>>>> Hi altogether,
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I'm new to this list and hope it's the right
>> >>>>>>>>>>>>> place to
>> >>>>>>>>>>>>> ask
>> >>>>>>>>>>>>> this
>> >>>>>>>>> question.
>> >>>>>>>>>>>>> If not, and someone know the right place to ask
>> >>>>>>>>>>>>> this,
>> >>>>>>>>>>>>> please give
>> >>>>>>>>> advise.
>> >>>>>>>>>>>>> I'm developing a big JSF Application for a
>> >>>>>>>>>>>>> customer.
>> >>>>>>>>>>>>> It's
>> >>>>>>>>>>>>> a long
>> >>>>>>>>> running
>> >>>>>>>>>>>>> project and development and when I started with
>> >>>>>>>>>>>>> it,
>> >>>>>>>>>>>>> TomEE
>> >>>>>>>>>>>>> 8.0.6 was
>> >>>>>>>>> the
>> >>>>>>>>>>>>> most recent Version of TomEE. The switch to
>> >>>>>>>>>>>>> Jakarta
>> >>>>>>>>>>>>> is
>> >>>>>>>>>>>>> planned for
>> >>>>>>>>> next
>> >>>>>>>>>>>>> year, so I'm still on the 8.x path for the time
>> >>>>>>>>>>>>> being
>> >>>>>>>>>>>>> and
>> >>>>>>>>>>>>> wasn't able
>> >>>>>>>>> to
>> >>>>>>>>>>>>> test if this problem still exists in the 9.x
>> >>>>>>>>>>>>> branch.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> The problem:
>> >>>>>>>>>>>>> On the customers site I'm with the latest TomEE
>> >>>>>>>>>>>>> version
>> >>>>>>>>>>>>> 8.0.15
>> >>>>>>>>> (running
>> >>>>>>>>>>>>> on Java 11) and everything works fine, because
>> >>>>>>>>>>>>> it is
>> >>>>>>>>>>>>> running in
>> >>>>>>>>>>>>> production mode. But in my development
>> >>>>>>>>>>>>> environment
>> >>>>>>>>>>>>> I'm
>> >>>>>>>>>>>>> still stuck
>> >>>>>>>>> with
>> >>>>>>>>>>>>> 8.0.6, because there I need the development
>> >>>>>>>>>>>>> mode
>> >>>>>>>>>>>>> (javax.faces.PROJECT_STAGE: Development). I
>> >>>>>>>>>>>>> need to
>> >>>>>>>>>>>>> be
>> >>>>>>>>>>>>> able to see
>> >>>>>>>>>>>>> changes in facelets without restarting
>> >>>>>>>>>>>>> everything
>> >>>>>>>>>>>>> everytime, and to
>> >>>>>>>>> get
>> >>>>>>>>>>>>> more detailed error messages.
>> >>>>>>>>>>>>> My attempts to update my dev TomEE to something
>> >>>>>>>>>>>>> newer
>> >>>>>>>>>>>>> than 8.0.6 all
>> >>>>>>>>>>>>> failed, because all (at least all I've tested
>> >>>>>>>>>>>>> so far)
>> >>>>>>>>>>>>> newer versions
>> >>>>>>>>> are
>> >>>>>>>>>>>>> so incredibly slow in dev mode, that it's
>> >>>>>>>>>>>>> unbearable
>> >>>>>>>>>>>>> to
>> >>>>>>>>>>>>> use the
>> >>>>>>>>>>>>> application. Several very looong seconds on
>> >>>>>>>>>>>>> every
>> >>>>>>>>>>>>> request
>> >>>>>>>>>>>>> (even the
>> >>>>>>>>>>>>> little AJAX requests in a view) is simply not a
>> >>>>>>>>>>>>> practical
>> >>>>>>>>>>>>> environment.
>> >>>>>>>>>>>>> After switching to Production mode everything
>> >>>>>>>>>>>>> works
>> >>>>>>>>>>>>> very
>> >>>>>>>>>>>>> fast, and all
>> >>>>>>>>>>>>> my views have response times of very few ms
>> >>>>>>>>>>>>> (even the
>> >>>>>>>>>>>>> big
>> >>>>>>>>>>>>> ones with
>> >>>>>>>>> max.
>> >>>>>>>>>>>>> around 400ms). When switching back to
>> >>>>>>>>>>>>> Development
>> >>>>>>>>>>>>> mode I
>> >>>>>>>>>>>>> have response
>> >>>>>>>>>>>>> times of up to 30s on big views. Not with
>> >>>>>>>>>>>>> 8.0.6,
>> >>>>>>>>>>>>> there
>> >>>>>>>>>>>>> even in
>> >>>>>>>>>>>>> development mode it's reasonable fast in not
>> >>>>>>>>>>>>> getting
>> >>>>>>>>>>>>> higher than 1s.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> My question:
>> >>>>>>>>>>>>> What causes this enormous performance
>> >>>>>>>>>>>>> degradation?
>> >>>>>>>>>>>>> I'm
>> >>>>>>>>>>>>> hoping, it is
>> >>>>>>>>>>>>> just a configuration which now has another
>> >>>>>>>>>>>>> default
>> >>>>>>>>>>>>> value
>> >>>>>>>>>>>>> or the like.
>> >>>>>>>>> If
>> >>>>>>>>>>>>> yes, maybe someone can point me in the right
>> >>>>>>>>>>>>> direction of
>> >>>>>>>>>>>>> where to
>> >>>>>>>>> find
>> >>>>>>>>>>>>> this configuration? If it's not a configuration
>> >>>>>>>>>>>>> thing:
>> >>>>>>>>>>>>> what can I do
>> >>>>>>>>> to
>> >>>>>>>>>>>>> get around this?
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I haven't tested all Versions of TomEE from
>> >>>>>>>>>>>>> 8.0.6 to
>> >>>>>>>>>>>>> 8.0.15, so I
>> >>>>>>>>> can't
>> >>>>>>>>>>>>> say at what version exactly this behaviour
>> >>>>>>>>>>>>> changes.
>> >>>>>>>>>>>>> If it
>> >>>>>>>>>>>>> helps or is
>> >>>>>>>>>>>>> needed, I can make some tests to find the exact
>> >>>>>>>>>>>>> version
>> >>>>>>>>>>>>> where this
>> >>>>>>>>>>>>> happens. Versions I've already tested are:
>> >>>>>>>>>>>>> 8.0.12,
>> >>>>>>>>>>>>> 8.0.13
>> >>>>>>>>>>>>> and 8.0.15.
>> >>>>>>>>>>>>> All of them are very slow in dev mode. If you
>> >>>>>>>>>>>>> need
>> >>>>>>>>>>>>> more
>> >>>>>>>>>>>>> information,
>> >>>>>>>>>>>>> I'll try to provide it.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Thanks in advance for any help
>> >>>>>>>>>>>>> cu
>> >>>>>>>>>>>>> Jens
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>>
>>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Vicente Rossello <co...@gmail.com>.
Hi,
It is a tomcat problem. The real problem is in DirResourcesSet. It is
calling a lot of times File.getCanonicalPath(). This method is very slow in
windows.
What I did is extending the DirResourcesSet to skip processing the
getCanonicalPath method so many times, but I really didn't understand the
problem.
A simpler and better solution is to set allowLinking="true" in the
context.xml file. So tomcat does not validate the canonicalPath every
single time, which is just too much.
Best regards,
Vicente
On Fri, Oct 27, 2023 at 5:51 PM Jens Zurawski <jz...@diegurus.de> wrote:
> My client is the last Windows System in my little universe. Due to some
> constraints with my customers I have to deal with it for the time being.
>
> The switches, Vicente has provided, solved the problem for me so far.
> But my "inner monk" is still not satisfied, because practical no one
> knows of these switches, and after all, TomEE should do a good job on
> Windows, too. And because 8.0.6 does a good job, and 8.0.8 breaks it to
> some degree, we should at least try to figure out, what exactly causes
> this break. Maybe there is an easy way to fix it, then it should be
> fixed IMHO.
>
> So, I will go through the bisect session anyway, as it seems the best
> possibility to get a grip on it. But I don't want to make these builds
> on Windows. I'll build them on Linux and will try to ease my life a bit
> with some scripting.
>
> cu
> Jens
>
>
> Am 27.10.2023 um 17:11 schrieb Richard Zowalla:
> > Great news! (though I am not a Windows user anymore).
> >
> > You should be able to build TomEE on Windows, too. If that does not
> > work out-of-the-box, you could try to build within a docker container
> > or via git-bash in Windows
> >
> > (I am not 100% sure a Windows build will work out of the box)
> >
> > I agree, that it might complicate things for testing purpose as it
> > increases the amount of cycles needed, but that are already very good
> > insights!
> >
> > Gruß
> > Richard
> >
> >
> > Am Freitag, dem 27.10.2023 um 16:55 +0200 schrieb Jens Zurawski:
> >> I have a first result on this issue:
> >>
> >> The problem does not occur if TomEE is running on Linux. I've tested
> >> my
> >> own built 8.0.8, a freshly extracted 8.0.8 and 8.0.15 from the
> >> releases,
> >> and none of them showed the dev mode performance problem.
> >>
> >> Using TomEE 8.0.8 on Windows does show the dev mode performance
> >> problem.
> >> All of the tested ones, means the released 8.0.8 and my own built
> >> 8.0.8
> >> (the one I've built on Linux).
> >>
> >> My development environment is Windows 10 (Netbeans and therefore also
> >> the TomEE for easy deployment and debugging). The problem does occur
> >> on
> >> stand alone started TomEE and also if started out of Netbeans, so
> >> Netbeans can be sorted out as the root of the problem. Java Versions
> >> 11,
> >> 13, 15 and 17 tested, all the same, so Java should also not be the
> >> problem. BTW: It is very unlikely a problem of my individual Windows
> >> installation, because this problem also shows up on the Windows
> >> system
> >> of my colleague, which btw. was the root cause why I'm started to dig
> >> deeper into this problem now and why I subscribed to this mailing
> >> list ;-)
> >>
> >> I'm afraid, this will complicate my git bisect session a little bit
> >> :-(
> >>
> >> However, I'll keep you posted.
> >>
> >> cu
> >> Jens
> >>
> >>
> >> Am 27.10.2023 um 13:12 schrieb Jens Zurawski:
> >>> Thanks Richard (and Vicente),
> >>>
> >>> I've changed to JDK 8 and now the builds are successful (both 8.0.6
> >>> and 8.0.8).
> >>>
> >>> I'll script me some test scripts to ease the build, run and test
> >>> for
> >>> every bisect step.
> >>> A build needs around 3 to 4 minutes. Running a test with my
> >>> application will need some manual interaction and checks, I guess
> >>> also
> >>> something around 4 minutes. This sums up to roughly 8 minutes per
> >>> step.
> >>>
> >>> I've started a bisect and it told me:
> >>> git bisect good 20441eb
> >>> git bisect bad 4c8a616
> >>> Bisecting: 136 revisions left to test after this (roughly 7 steps)
> >>>
> >>> That means, if everything works fine, I should be able to identify
> >>> the
> >>> commit after about an hour? (That's cool.... why nobody has told me
> >>> about git bisect before? ;-) )
> >>>
> >>> Anyway, I will have to find some time for this, maybe I need a
> >>> couple
> >>> of days to find some.
> >>> I will let you know about the outcome.
> >>>
> >>> cu
> >>> Jens
> >>>
> >>>
> >>> Am 27.10.2023 um 11:50 schrieb Richard Zowalla:
> >>>> Hi,
> >>>>
> >>>> you need Java/JDK 8 to build TomEE 8.
> >>>>
> >>>> The Maven command is
> >>>>
> >>>> mvn --fail-at-end clean install -pl tomee/apache-tomee -am -
> >>>> Dfile.encoding=UTF-8 -DskipTests=true
> >>>>
> >>>> You will find the tar.gz/zip in ./tomee/apache-tomee/target
> >>>>
> >>>> Gruß
> >>>> Richard
> >>>>
> >>>> P.S. 8.0.7 is here:
> >>>> https://archive.apache.org/dist/tomee/tomee-8.0.7/
> >>>>
> >>>> Am Freitag, dem 27.10.2023 um 10:50 +0200 schrieb Jens Zurawski:
> >>>>> Hi Richard,
> >>>>>
> >>>>> to identify the "bad" version was easy. I didn't find a 8.0.7
> >>>>> release
> >>>>> on
> >>>>> the download page, so I started with 8.0.8. And this version
> >>>>> already
> >>>>> has
> >>>>> the slow dev mode. So for my problem I can say:
> >>>>> good: 8.0.6
> >>>>> bad: 8.0.8
> >>>>>
> >>>>> Now, I think, I need a little help from you. I tried to build a
> >>>>> TomEE
> >>>>> out of git (I've cloned the github repository
> >>>>> https://github.com/apache/tomee.git). I've tried these two
> >>>>> commits:
> >>>>> 20441eb (tag: tomee-8.0.6)
> >>>>> 4c8a616 (tag: tomee-project-8.0.8)
> >>>>>
> >>>>> But the build fails every time on "TomEE :: Container :: Core"
> >>>>> with:
> >>>>>
> >>>>> [INFO] BUILD FAILURE
> >>>>> [INFO]
> >>>>> ---------------------------------------------------------------
> >>>>> ------
> >>>>> ---
> >>>>> [INFO] Total time: 01:19 min
> >>>>> [INFO] Finished at: 2023-10-27T10:29:40+02:00
> >>>>> [INFO]
> >>>>> ---------------------------------------------------------------
> >>>>> ------
> >>>>> ---
> >>>>> [ERROR] Failed to execute goal
> >>>>> org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile
> >>>>> (default-compile) on project openejb-core: Compilation failure:
> >>>>> Compilation failure:
> >>>>> [ERROR]
> >>>>> /sandbox/tomee/container/openejb-
> >>>>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[21,
> >>>>> 21]
> >>>>> package org.omg.CORBA does not exist
> >>>>> [ERROR]
> >>>>> /sandbox/tomee/container/openejb-
> >>>>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[24,
> >>>>> 12]
> >>>>> cannot find symbol
> >>>>> [ERROR] symbol: class ORB
> >>>>> [ERROR] location: class org.apache.openejb.core.OrbFactory
> >>>>>
> >>>>> I've tried to build with:
> >>>>> mvn clean install
> >>>>>
> >>>>> My build environment is a linux ubuntu 20.04.6 LTS VM with:
> >>>>> git 2.25.1
> >>>>> Maven 3.6.3
> >>>>> java 14.0.2
> >>>>>
> >>>>> Is it some easy thing to fix, or do I have to switch some
> >>>>> versions of
> >>>>> my
> >>>>> environment?
> >>>>>
> >>>>> cu
> >>>>> Jens
> >>>>>
> >>>>> Am 26.10.2023 um 19:48 schrieb Richard Zowalla:
> >>>>>> Hi,
> >>>>>>
> >>>>>> without knowing code or having a reproducer, it might be not
> >>>>>> that
> >>>>>> obvious to find a reason / answer for the "academic problem"
> >>>>>> ;-)
> >>>>>>
> >>>>>> If you really want to boil it down, I recommend the following
> >>>>>> approach:
> >>>>>>
> >>>>>> (1) From your description it seems, that 8.0.6 is the last
> >>>>>> known
> >>>>>> "good"
> >>>>>> version.
> >>>>>>
> >>>>>> (2) You've investigated versions starting from 8.0.12
> >>>>>> onwards,
> >>>>>> which
> >>>>>> seems to be "bad".
> >>>>>>
> >>>>>> Following the approach recommended by Jonathan:
> >>>>>>
> >>>>>> - Can you check if it happens in 8.0.7 (8.0.8, ...) until we
> >>>>>> find
> >>>>>> the
> >>>>>> first "bad" released version?
> >>>>>>
> >>>>>> - If we have that, you could start a "git bisect" session
> >>>>>> between
> >>>>>> 8.0.6
> >>>>>> and the first version, which is known to be bad (reducing the
> >>>>>> amount of
> >>>>>> needed bisect cycles dramatically), so we find a commit /
> >>>>>> reason
> >>>>>> (if it
> >>>>>> is in TomEE).
> >>>>>>
> >>>>>> - It would involve some quick builds without executing tests
> >>>>>> (~10-
> >>>>>> 15min
> >>>>>> each) for each bisect step and testing on your side.
> >>>>>>
> >>>>>> Don't know if your inner monk is willing to go that route,
> >>>>>> but I am
> >>>>>> happy to help with the related Maven commands for quick
> >>>>>> building :)
> >>>>>>
> >>>>>> Gruß
> >>>>>> Richard
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Am Donnerstag, dem 26.10.2023 um 18:00 +0200 schrieb Jens
> >>>>>> Zurawski:
> >>>>>>> Hi Vicente,
> >>>>>>>
> >>>>>>> thank you for the tip. This is a good workaround. When I
> >>>>>>> switch
> >>>>>>> to
> >>>>>>> Production mode and insert this configuration, it reloads
> >>>>>>> the
> >>>>>>> view
> >>>>>>> from
> >>>>>>> the file if it has changed. This means, I can use a decent
> >>>>>>> version of
> >>>>>>> TomEE in my dev environment.
> >>>>>>>
> >>>>>>> However, it doesn't completely solve my problem, as the
> >>>>>>> error
> >>>>>>> reporting
> >>>>>>> isn't as verbose as in Dev mode as far as I can see. But I
> >>>>>>> have
> >>>>>>> to
> >>>>>>> check, maybe it's enough for my development.
> >>>>>>>
> >>>>>>> And, well.... the academic problem: "Why is it still broken
> >>>>>>> in
> >>>>>>> Dev
> >>>>>>> mode?" still runs circles through my brain ;-) Because if
> >>>>>>> it's a
> >>>>>>> problem
> >>>>>>> of my coding, it might be a good idea to change this,
> >>>>>>> before it
> >>>>>>> might
> >>>>>>> get a problem on Production mode, too, at some time in the
> >>>>>>> future.
> >>>>>>>
> >>>>>>> But anyway, you've helped me a good step forward, thanks.
> >>>>>>> cu
> >>>>>>> Jens
> >>>>>>>
> >>>>>>>
> >>>>>>> Am 26.10.2023 um 17:04 schrieb Vicente Rossello:
> >>>>>>>> We are using the production stage with
> >>>>>>>> javax.faces.FACELETS_REFRESH_PERIOD
> >>>>>>>> = 1. It's faster and more similar to production, just try
> >>>>>>>> it if
> >>>>>>>> it
> >>>>>>>> solves
> >>>>>>>> your problem
> >>>>>>>>
> >>>>>>>> On Thu, Oct 26, 2023 at 4:18 PM Jens Zurawski
> >>>>>>>> <jz...@diegurus.de>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> The one bundled with TomEE.
> >>>>>>>>> myfaces 2.3.7 on TomEE 8.0.6
> >>>>>>>>> and e.g. myfaces 2.3.9 on TomEE 8.0.12
> >>>>>>>>>
> >>>>>>>>> Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
> >>>>>>>>>> Which JSF lib are you using?
> >>>>>>>>>>
> >>>>>>>>>> Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens
> >>>>>>>>>> Zurawski
> >>>>>>>>>> <jzu@diegurus.de
> >>>>>>>>>> :
> >>>>>>>>>>> Thank you for your quick reply, Jonathan.
> >>>>>>>>>>>
> >>>>>>>>>>> It is good to know, that normally the Dev mode
> >>>>>>>>>>> should
> >>>>>>>>>>> work as
> >>>>>>>>>>> fast as
> >>>>>>>>> before even with 8.0.15. So it obviously has something
> >>>>>>>>> to do
> >>>>>>>>> with
> >>>>>>>>> my
> >>>>>>>>> environment, configuration and/or code.
> >>>>>>>>>>> And also thank you for the reference to tomitribe.
> >>>>>>>>>>> Unfortunately I'm
> >>>>>>>>> the only developer on this project, and it will be
> >>>>>>>>> difficult
> >>>>>>>>> to
> >>>>>>>>> convince my
> >>>>>>>>> customer to an additional support subscription, because
> >>>>>>>>> he on
> >>>>>>>>> production
> >>>>>>>>> doesn't have issues ;-) But, ok, that's my problem, and
> >>>>>>>>> maybe
> >>>>>>>>> I'll consult
> >>>>>>>>> tomitribe in the future for other projects.
> >>>>>>>>>>> So, I'm afraid, for the near future I'm on my own
> >>>>>>>>>>> to
> >>>>>>>>>>> solve
> >>>>>>>>>>> this
> >>>>>>>>> problem. Therefore any hints to restrict the places
> >>>>>>>>> where I
> >>>>>>>>> should take a
> >>>>>>>>> look will be more than welcome.
> >>>>>>>>>>> I'm a senior programmer with 20 years of experience
> >>>>>>>>>>> in
> >>>>>>>>>>> Java
> >>>>>>>>>>> programming
> >>>>>>>>> (SE and JSP). But I'm relative new to
> >>>>>>>>> JSF/JavaEE/JakartaEE
> >>>>>>>>> and
> >>>>>>>>> sometimes I
> >>>>>>>>> have a hard time to understand which component of the
> >>>>>>>>> whole
> >>>>>>>>> stack
> >>>>>>>>> is
> >>>>>>>>> responsible for what.
> >>>>>>>>>>> The project meanwhile has over a 1/4 million lines
> >>>>>>>>>>> of
> >>>>>>>>>>> code
> >>>>>>>>>>> (most of
> >>>>>>>>> them in the back-end, but also the front-end is big
> >>>>>>>>> enough to
> >>>>>>>>> not
> >>>>>>>>> trace
> >>>>>>>>> through every lane if I want to get to the point in
> >>>>>>>>> this life
> >>>>>>>>> :-D
> >>>>>>>>> ).
> >>>>>>>>>>> The application now runs for nearly 2 years on the
> >>>>>>>>>>> customers
> >>>>>>>>>>> site and
> >>>>>>>>> is very reliable and fast in its daily usage. Apart
> >>>>>>>>> from
> >>>>>>>>> occasional updates
> >>>>>>>>> every few month, it will never be restarted or need any
> >>>>>>>>> manual
> >>>>>>>>> maintenance
> >>>>>>>>> at all. So I assume the code is not the badest on
> >>>>>>>>> earth. At
> >>>>>>>>> least
> >>>>>>>>> because
> >>>>>>>>> of that I'm able to out-rule things like memory leaks,
> >>>>>>>>> gc
> >>>>>>>>> issues
> >>>>>>>>> or
> >>>>>>>>> bottlenecks (max Heap is never reached, neither half of
> >>>>>>>>> it,
> >>>>>>>>> even
> >>>>>>>>> after
> >>>>>>>>> running for month).
> >>>>>>>>>>> Of course there could be aspects of JSF which I may
> >>>>>>>>>>> not
> >>>>>>>>>>> have
> >>>>>>>>>>> understood
> >>>>>>>>> correctly until now. But because there are so many new
> >>>>>>>>> things
> >>>>>>>>> to
> >>>>>>>>> a JSF
> >>>>>>>>> beginner like me, I'm running out of ideas what this
> >>>>>>>>> could
> >>>>>>>>> be. It
> >>>>>>>>> is
> >>>>>>>>> something that worked well 'til TomEE 8.0.6 (Prod and
> >>>>>>>>> Dev
> >>>>>>>>> mode)
> >>>>>>>>> and
> >>>>>>>>> suddenly don't work anymore on 8.0.12 in Dev mode (but
> >>>>>>>>> still
> >>>>>>>>> works great in
> >>>>>>>>> Prod mode). There are only minor versions in-between.
> >>>>>>>>> So I
> >>>>>>>>> think
> >>>>>>>>> there were
> >>>>>>>>> no revolutionary changes anywhere. My hope was (and
> >>>>>>>>> still
> >>>>>>>>> is),
> >>>>>>>>> that anybody
> >>>>>>>>> here on the list might have had a similar problem and
> >>>>>>>>> has
> >>>>>>>>> solved
> >>>>>>>>> it, or
> >>>>>>>>> maybe someone knows about some changes in Dev mode
> >>>>>>>>> which
> >>>>>>>>> could
> >>>>>>>>> explain such
> >>>>>>>>> behaviour (don't know, maybe xml parsing or component
> >>>>>>>>> tree
> >>>>>>>>> generation has
> >>>>>>>>> changed, or scope handling of managed beans, or
> >>>>>>>>> whatever
> >>>>>>>>> could
> >>>>>>>>> cause some
> >>>>>>>>> code to be entered far more often than before, or some
> >>>>>>>>> cache/pool
> >>>>>>>>> was
> >>>>>>>>> removed or has different behaviour... such things).
> >>>>>>>>> Then it
> >>>>>>>>> would
> >>>>>>>>> be far
> >>>>>>>>> more easy for me to track down the real problem.
> >>>>>>>>>>> Anyway, when I get some spare time (currently have
> >>>>>>>>>>> some
> >>>>>>>>>>> deadlines in
> >>>>>>>>> sight), I'll try to build a small test case. Either I
> >>>>>>>>> can get
> >>>>>>>>> a
> >>>>>>>>> grip on the
> >>>>>>>>> issue myself on doing this, or I can supply it to
> >>>>>>>>> anyone who
> >>>>>>>>> might want to
> >>>>>>>>> have a look at it.
> >>>>>>>>>>> cu
> >>>>>>>>>>> Jens
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
> >>>>>>>>>>>>> newer versions are so incredibly slow in dev
> >>>>>>>>>>>>> mode
> >>>>>>>>>>>> I can assure you that Dev/Prod mode works very
> >>>>>>>>>>>> very
> >>>>>>>>>>>> swiftly
> >>>>>>>>>>>> on 8.0.15
> >>>>>>>>>>>> for sizable applications. For example, I have a
> >>>>>>>>>>>> giant
> >>>>>>>>>>>> application
> >>>>>>>>>>>> (thousands of LOC, 50+ jsf views) that has sub-
> >>>>>>>>>>>> 8ms
> >>>>>>>>>>>> response
> >>>>>>>>>>>> times
> >>>>>>>>>>>> (minus database time). More than likely they are
> >>>>>>>>>>>> doing
> >>>>>>>>>>>> something they
> >>>>>>>>>>>> should not be doing with JSF apis and are causing
> >>>>>>>>>>>> problems.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I would check the obvious things first with
> >>>>>>>>>>>> VisualVM:
> >>>>>>>>>>>> heap
> >>>>>>>>>>>> size, gc
> >>>>>>>>>>>> frequency, etc. You can also run their profiler
> >>>>>>>>>>>> and see
> >>>>>>>>>>>> if
> >>>>>>>>>>>> you can
> >>>>>>>>>>>> spot anything immediately obvious.
> >>>>>>>>>>>>
> >>>>>>>>>>>> After that, the easiest way to locate the code
> >>>>>>>>>>>> slowing
> >>>>>>>>>>>> you
> >>>>>>>>>>>> down is
> >>>>>>>>>>>> bisection. Cut half the code out, run, and
> >>>>>>>>>>>> continue
> >>>>>>>>>>>> cutting
> >>>>>>>>>>>> until you
> >>>>>>>>>>>> can locate the problematic code. Then after that,
> >>>>>>>>>>>> isolate a
> >>>>>>>>>>>> test case
> >>>>>>>>>>>> where flipping a boolean flag triggers the issue
> >>>>>>>>>>>> and
> >>>>>>>>>>>> post
> >>>>>>>>>>>> the results
> >>>>>>>>>>>> here.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I would get ahold of:
> >>>>>>>>>>>> https://www.tomitribe.com who are
> >>>>>>>>>>>> literal
> >>>>>>>>>>>> experts in this stuff. They offer professional
> >>>>>>>>>>>> services
> >>>>>>>>>>>> to
> >>>>>>>>>>>> handle
> >>>>>>>>>>>> exactly these situations and can work directly in
> >>>>>>>>>>>> your
> >>>>>>>>>>>> codebase to
> >>>>>>>>>>>> help you find and fix the problem.
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski
> >>>>>>>>>>>> <jz...@diegurus.de> wrote:
> >>>>>>>>>>>>> Hi altogether,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I'm new to this list and hope it's the right
> >>>>>>>>>>>>> place to
> >>>>>>>>>>>>> ask
> >>>>>>>>>>>>> this
> >>>>>>>>> question.
> >>>>>>>>>>>>> If not, and someone know the right place to ask
> >>>>>>>>>>>>> this,
> >>>>>>>>>>>>> please give
> >>>>>>>>> advise.
> >>>>>>>>>>>>> I'm developing a big JSF Application for a
> >>>>>>>>>>>>> customer.
> >>>>>>>>>>>>> It's
> >>>>>>>>>>>>> a long
> >>>>>>>>> running
> >>>>>>>>>>>>> project and development and when I started with
> >>>>>>>>>>>>> it,
> >>>>>>>>>>>>> TomEE
> >>>>>>>>>>>>> 8.0.6 was
> >>>>>>>>> the
> >>>>>>>>>>>>> most recent Version of TomEE. The switch to
> >>>>>>>>>>>>> Jakarta
> >>>>>>>>>>>>> is
> >>>>>>>>>>>>> planned for
> >>>>>>>>> next
> >>>>>>>>>>>>> year, so I'm still on the 8.x path for the time
> >>>>>>>>>>>>> being
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>> wasn't able
> >>>>>>>>> to
> >>>>>>>>>>>>> test if this problem still exists in the 9.x
> >>>>>>>>>>>>> branch.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> The problem:
> >>>>>>>>>>>>> On the customers site I'm with the latest TomEE
> >>>>>>>>>>>>> version
> >>>>>>>>>>>>> 8.0.15
> >>>>>>>>> (running
> >>>>>>>>>>>>> on Java 11) and everything works fine, because
> >>>>>>>>>>>>> it is
> >>>>>>>>>>>>> running in
> >>>>>>>>>>>>> production mode. But in my development
> >>>>>>>>>>>>> environment
> >>>>>>>>>>>>> I'm
> >>>>>>>>>>>>> still stuck
> >>>>>>>>> with
> >>>>>>>>>>>>> 8.0.6, because there I need the development
> >>>>>>>>>>>>> mode
> >>>>>>>>>>>>> (javax.faces.PROJECT_STAGE: Development). I
> >>>>>>>>>>>>> need to
> >>>>>>>>>>>>> be
> >>>>>>>>>>>>> able to see
> >>>>>>>>>>>>> changes in facelets without restarting
> >>>>>>>>>>>>> everything
> >>>>>>>>>>>>> everytime, and to
> >>>>>>>>> get
> >>>>>>>>>>>>> more detailed error messages.
> >>>>>>>>>>>>> My attempts to update my dev TomEE to something
> >>>>>>>>>>>>> newer
> >>>>>>>>>>>>> than 8.0.6 all
> >>>>>>>>>>>>> failed, because all (at least all I've tested
> >>>>>>>>>>>>> so far)
> >>>>>>>>>>>>> newer versions
> >>>>>>>>> are
> >>>>>>>>>>>>> so incredibly slow in dev mode, that it's
> >>>>>>>>>>>>> unbearable
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>> use the
> >>>>>>>>>>>>> application. Several very looong seconds on
> >>>>>>>>>>>>> every
> >>>>>>>>>>>>> request
> >>>>>>>>>>>>> (even the
> >>>>>>>>>>>>> little AJAX requests in a view) is simply not a
> >>>>>>>>>>>>> practical
> >>>>>>>>>>>>> environment.
> >>>>>>>>>>>>> After switching to Production mode everything
> >>>>>>>>>>>>> works
> >>>>>>>>>>>>> very
> >>>>>>>>>>>>> fast, and all
> >>>>>>>>>>>>> my views have response times of very few ms
> >>>>>>>>>>>>> (even the
> >>>>>>>>>>>>> big
> >>>>>>>>>>>>> ones with
> >>>>>>>>> max.
> >>>>>>>>>>>>> around 400ms). When switching back to
> >>>>>>>>>>>>> Development
> >>>>>>>>>>>>> mode I
> >>>>>>>>>>>>> have response
> >>>>>>>>>>>>> times of up to 30s on big views. Not with
> >>>>>>>>>>>>> 8.0.6,
> >>>>>>>>>>>>> there
> >>>>>>>>>>>>> even in
> >>>>>>>>>>>>> development mode it's reasonable fast in not
> >>>>>>>>>>>>> getting
> >>>>>>>>>>>>> higher than 1s.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> My question:
> >>>>>>>>>>>>> What causes this enormous performance
> >>>>>>>>>>>>> degradation?
> >>>>>>>>>>>>> I'm
> >>>>>>>>>>>>> hoping, it is
> >>>>>>>>>>>>> just a configuration which now has another
> >>>>>>>>>>>>> default
> >>>>>>>>>>>>> value
> >>>>>>>>>>>>> or the like.
> >>>>>>>>> If
> >>>>>>>>>>>>> yes, maybe someone can point me in the right
> >>>>>>>>>>>>> direction of
> >>>>>>>>>>>>> where to
> >>>>>>>>> find
> >>>>>>>>>>>>> this configuration? If it's not a configuration
> >>>>>>>>>>>>> thing:
> >>>>>>>>>>>>> what can I do
> >>>>>>>>> to
> >>>>>>>>>>>>> get around this?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I haven't tested all Versions of TomEE from
> >>>>>>>>>>>>> 8.0.6 to
> >>>>>>>>>>>>> 8.0.15, so I
> >>>>>>>>> can't
> >>>>>>>>>>>>> say at what version exactly this behaviour
> >>>>>>>>>>>>> changes.
> >>>>>>>>>>>>> If it
> >>>>>>>>>>>>> helps or is
> >>>>>>>>>>>>> needed, I can make some tests to find the exact
> >>>>>>>>>>>>> version
> >>>>>>>>>>>>> where this
> >>>>>>>>>>>>> happens. Versions I've already tested are:
> >>>>>>>>>>>>> 8.0.12,
> >>>>>>>>>>>>> 8.0.13
> >>>>>>>>>>>>> and 8.0.15.
> >>>>>>>>>>>>> All of them are very slow in dev mode. If you
> >>>>>>>>>>>>> need
> >>>>>>>>>>>>> more
> >>>>>>>>>>>>> information,
> >>>>>>>>>>>>> I'll try to provide it.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks in advance for any help
> >>>>>>>>>>>>> cu
> >>>>>>>>>>>>> Jens
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
>
>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Jens Zurawski <jz...@diegurus.de>.
My client is the last Windows System in my little universe. Due to some
constraints with my customers I have to deal with it for the time being.
The switches, Vicente has provided, solved the problem for me so far.
But my "inner monk" is still not satisfied, because practical no one
knows of these switches, and after all, TomEE should do a good job on
Windows, too. And because 8.0.6 does a good job, and 8.0.8 breaks it to
some degree, we should at least try to figure out, what exactly causes
this break. Maybe there is an easy way to fix it, then it should be
fixed IMHO.
So, I will go through the bisect session anyway, as it seems the best
possibility to get a grip on it. But I don't want to make these builds
on Windows. I'll build them on Linux and will try to ease my life a bit
with some scripting.
cu
Jens
Am 27.10.2023 um 17:11 schrieb Richard Zowalla:
> Great news! (though I am not a Windows user anymore).
>
> You should be able to build TomEE on Windows, too. If that does not
> work out-of-the-box, you could try to build within a docker container
> or via git-bash in Windows
>
> (I am not 100% sure a Windows build will work out of the box)
>
> I agree, that it might complicate things for testing purpose as it
> increases the amount of cycles needed, but that are already very good
> insights!
>
> Gruß
> Richard
>
>
> Am Freitag, dem 27.10.2023 um 16:55 +0200 schrieb Jens Zurawski:
>> I have a first result on this issue:
>>
>> The problem does not occur if TomEE is running on Linux. I've tested
>> my
>> own built 8.0.8, a freshly extracted 8.0.8 and 8.0.15 from the
>> releases,
>> and none of them showed the dev mode performance problem.
>>
>> Using TomEE 8.0.8 on Windows does show the dev mode performance
>> problem.
>> All of the tested ones, means the released 8.0.8 and my own built
>> 8.0.8
>> (the one I've built on Linux).
>>
>> My development environment is Windows 10 (Netbeans and therefore also
>> the TomEE for easy deployment and debugging). The problem does occur
>> on
>> stand alone started TomEE and also if started out of Netbeans, so
>> Netbeans can be sorted out as the root of the problem. Java Versions
>> 11,
>> 13, 15 and 17 tested, all the same, so Java should also not be the
>> problem. BTW: It is very unlikely a problem of my individual Windows
>> installation, because this problem also shows up on the Windows
>> system
>> of my colleague, which btw. was the root cause why I'm started to dig
>> deeper into this problem now and why I subscribed to this mailing
>> list ;-)
>>
>> I'm afraid, this will complicate my git bisect session a little bit
>> :-(
>>
>> However, I'll keep you posted.
>>
>> cu
>> Jens
>>
>>
>> Am 27.10.2023 um 13:12 schrieb Jens Zurawski:
>>> Thanks Richard (and Vicente),
>>>
>>> I've changed to JDK 8 and now the builds are successful (both 8.0.6
>>> and 8.0.8).
>>>
>>> I'll script me some test scripts to ease the build, run and test
>>> for
>>> every bisect step.
>>> A build needs around 3 to 4 minutes. Running a test with my
>>> application will need some manual interaction and checks, I guess
>>> also
>>> something around 4 minutes. This sums up to roughly 8 minutes per
>>> step.
>>>
>>> I've started a bisect and it told me:
>>> git bisect good 20441eb
>>> git bisect bad 4c8a616
>>> Bisecting: 136 revisions left to test after this (roughly 7 steps)
>>>
>>> That means, if everything works fine, I should be able to identify
>>> the
>>> commit after about an hour? (That's cool.... why nobody has told me
>>> about git bisect before? ;-) )
>>>
>>> Anyway, I will have to find some time for this, maybe I need a
>>> couple
>>> of days to find some.
>>> I will let you know about the outcome.
>>>
>>> cu
>>> Jens
>>>
>>>
>>> Am 27.10.2023 um 11:50 schrieb Richard Zowalla:
>>>> Hi,
>>>>
>>>> you need Java/JDK 8 to build TomEE 8.
>>>>
>>>> The Maven command is
>>>>
>>>> mvn --fail-at-end clean install -pl tomee/apache-tomee -am -
>>>> Dfile.encoding=UTF-8 -DskipTests=true
>>>>
>>>> You will find the tar.gz/zip in ./tomee/apache-tomee/target
>>>>
>>>> Gruß
>>>> Richard
>>>>
>>>> P.S. 8.0.7 is here:
>>>> https://archive.apache.org/dist/tomee/tomee-8.0.7/
>>>>
>>>> Am Freitag, dem 27.10.2023 um 10:50 +0200 schrieb Jens Zurawski:
>>>>> Hi Richard,
>>>>>
>>>>> to identify the "bad" version was easy. I didn't find a 8.0.7
>>>>> release
>>>>> on
>>>>> the download page, so I started with 8.0.8. And this version
>>>>> already
>>>>> has
>>>>> the slow dev mode. So for my problem I can say:
>>>>> good: 8.0.6
>>>>> bad: 8.0.8
>>>>>
>>>>> Now, I think, I need a little help from you. I tried to build a
>>>>> TomEE
>>>>> out of git (I've cloned the github repository
>>>>> https://github.com/apache/tomee.git). I've tried these two
>>>>> commits:
>>>>> 20441eb (tag: tomee-8.0.6)
>>>>> 4c8a616 (tag: tomee-project-8.0.8)
>>>>>
>>>>> But the build fails every time on "TomEE :: Container :: Core"
>>>>> with:
>>>>>
>>>>> [INFO] BUILD FAILURE
>>>>> [INFO]
>>>>> ---------------------------------------------------------------
>>>>> ------
>>>>> ---
>>>>> [INFO] Total time: 01:19 min
>>>>> [INFO] Finished at: 2023-10-27T10:29:40+02:00
>>>>> [INFO]
>>>>> ---------------------------------------------------------------
>>>>> ------
>>>>> ---
>>>>> [ERROR] Failed to execute goal
>>>>> org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile
>>>>> (default-compile) on project openejb-core: Compilation failure:
>>>>> Compilation failure:
>>>>> [ERROR]
>>>>> /sandbox/tomee/container/openejb-
>>>>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[21,
>>>>> 21]
>>>>> package org.omg.CORBA does not exist
>>>>> [ERROR]
>>>>> /sandbox/tomee/container/openejb-
>>>>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[24,
>>>>> 12]
>>>>> cannot find symbol
>>>>> [ERROR] symbol: class ORB
>>>>> [ERROR] location: class org.apache.openejb.core.OrbFactory
>>>>>
>>>>> I've tried to build with:
>>>>> mvn clean install
>>>>>
>>>>> My build environment is a linux ubuntu 20.04.6 LTS VM with:
>>>>> git 2.25.1
>>>>> Maven 3.6.3
>>>>> java 14.0.2
>>>>>
>>>>> Is it some easy thing to fix, or do I have to switch some
>>>>> versions of
>>>>> my
>>>>> environment?
>>>>>
>>>>> cu
>>>>> Jens
>>>>>
>>>>> Am 26.10.2023 um 19:48 schrieb Richard Zowalla:
>>>>>> Hi,
>>>>>>
>>>>>> without knowing code or having a reproducer, it might be not
>>>>>> that
>>>>>> obvious to find a reason / answer for the "academic problem"
>>>>>> ;-)
>>>>>>
>>>>>> If you really want to boil it down, I recommend the following
>>>>>> approach:
>>>>>>
>>>>>> (1) From your description it seems, that 8.0.6 is the last
>>>>>> known
>>>>>> "good"
>>>>>> version.
>>>>>>
>>>>>> (2) You've investigated versions starting from 8.0.12
>>>>>> onwards,
>>>>>> which
>>>>>> seems to be "bad".
>>>>>>
>>>>>> Following the approach recommended by Jonathan:
>>>>>>
>>>>>> - Can you check if it happens in 8.0.7 (8.0.8, ...) until we
>>>>>> find
>>>>>> the
>>>>>> first "bad" released version?
>>>>>>
>>>>>> - If we have that, you could start a "git bisect" session
>>>>>> between
>>>>>> 8.0.6
>>>>>> and the first version, which is known to be bad (reducing the
>>>>>> amount of
>>>>>> needed bisect cycles dramatically), so we find a commit /
>>>>>> reason
>>>>>> (if it
>>>>>> is in TomEE).
>>>>>>
>>>>>> - It would involve some quick builds without executing tests
>>>>>> (~10-
>>>>>> 15min
>>>>>> each) for each bisect step and testing on your side.
>>>>>>
>>>>>> Don't know if your inner monk is willing to go that route,
>>>>>> but I am
>>>>>> happy to help with the related Maven commands for quick
>>>>>> building :)
>>>>>>
>>>>>> Gruß
>>>>>> Richard
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Am Donnerstag, dem 26.10.2023 um 18:00 +0200 schrieb Jens
>>>>>> Zurawski:
>>>>>>> Hi Vicente,
>>>>>>>
>>>>>>> thank you for the tip. This is a good workaround. When I
>>>>>>> switch
>>>>>>> to
>>>>>>> Production mode and insert this configuration, it reloads
>>>>>>> the
>>>>>>> view
>>>>>>> from
>>>>>>> the file if it has changed. This means, I can use a decent
>>>>>>> version of
>>>>>>> TomEE in my dev environment.
>>>>>>>
>>>>>>> However, it doesn't completely solve my problem, as the
>>>>>>> error
>>>>>>> reporting
>>>>>>> isn't as verbose as in Dev mode as far as I can see. But I
>>>>>>> have
>>>>>>> to
>>>>>>> check, maybe it's enough for my development.
>>>>>>>
>>>>>>> And, well.... the academic problem: "Why is it still broken
>>>>>>> in
>>>>>>> Dev
>>>>>>> mode?" still runs circles through my brain ;-) Because if
>>>>>>> it's a
>>>>>>> problem
>>>>>>> of my coding, it might be a good idea to change this,
>>>>>>> before it
>>>>>>> might
>>>>>>> get a problem on Production mode, too, at some time in the
>>>>>>> future.
>>>>>>>
>>>>>>> But anyway, you've helped me a good step forward, thanks.
>>>>>>> cu
>>>>>>> Jens
>>>>>>>
>>>>>>>
>>>>>>> Am 26.10.2023 um 17:04 schrieb Vicente Rossello:
>>>>>>>> We are using the production stage with
>>>>>>>> javax.faces.FACELETS_REFRESH_PERIOD
>>>>>>>> = 1. It's faster and more similar to production, just try
>>>>>>>> it if
>>>>>>>> it
>>>>>>>> solves
>>>>>>>> your problem
>>>>>>>>
>>>>>>>> On Thu, Oct 26, 2023 at 4:18 PM Jens Zurawski
>>>>>>>> <jz...@diegurus.de>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> The one bundled with TomEE.
>>>>>>>>> myfaces 2.3.7 on TomEE 8.0.6
>>>>>>>>> and e.g. myfaces 2.3.9 on TomEE 8.0.12
>>>>>>>>>
>>>>>>>>> Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
>>>>>>>>>> Which JSF lib are you using?
>>>>>>>>>>
>>>>>>>>>> Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens
>>>>>>>>>> Zurawski
>>>>>>>>>> <jzu@diegurus.de
>>>>>>>>>> :
>>>>>>>>>>> Thank you for your quick reply, Jonathan.
>>>>>>>>>>>
>>>>>>>>>>> It is good to know, that normally the Dev mode
>>>>>>>>>>> should
>>>>>>>>>>> work as
>>>>>>>>>>> fast as
>>>>>>>>> before even with 8.0.15. So it obviously has something
>>>>>>>>> to do
>>>>>>>>> with
>>>>>>>>> my
>>>>>>>>> environment, configuration and/or code.
>>>>>>>>>>> And also thank you for the reference to tomitribe.
>>>>>>>>>>> Unfortunately I'm
>>>>>>>>> the only developer on this project, and it will be
>>>>>>>>> difficult
>>>>>>>>> to
>>>>>>>>> convince my
>>>>>>>>> customer to an additional support subscription, because
>>>>>>>>> he on
>>>>>>>>> production
>>>>>>>>> doesn't have issues ;-) But, ok, that's my problem, and
>>>>>>>>> maybe
>>>>>>>>> I'll consult
>>>>>>>>> tomitribe in the future for other projects.
>>>>>>>>>>> So, I'm afraid, for the near future I'm on my own
>>>>>>>>>>> to
>>>>>>>>>>> solve
>>>>>>>>>>> this
>>>>>>>>> problem. Therefore any hints to restrict the places
>>>>>>>>> where I
>>>>>>>>> should take a
>>>>>>>>> look will be more than welcome.
>>>>>>>>>>> I'm a senior programmer with 20 years of experience
>>>>>>>>>>> in
>>>>>>>>>>> Java
>>>>>>>>>>> programming
>>>>>>>>> (SE and JSP). But I'm relative new to
>>>>>>>>> JSF/JavaEE/JakartaEE
>>>>>>>>> and
>>>>>>>>> sometimes I
>>>>>>>>> have a hard time to understand which component of the
>>>>>>>>> whole
>>>>>>>>> stack
>>>>>>>>> is
>>>>>>>>> responsible for what.
>>>>>>>>>>> The project meanwhile has over a 1/4 million lines
>>>>>>>>>>> of
>>>>>>>>>>> code
>>>>>>>>>>> (most of
>>>>>>>>> them in the back-end, but also the front-end is big
>>>>>>>>> enough to
>>>>>>>>> not
>>>>>>>>> trace
>>>>>>>>> through every lane if I want to get to the point in
>>>>>>>>> this life
>>>>>>>>> :-D
>>>>>>>>> ).
>>>>>>>>>>> The application now runs for nearly 2 years on the
>>>>>>>>>>> customers
>>>>>>>>>>> site and
>>>>>>>>> is very reliable and fast in its daily usage. Apart
>>>>>>>>> from
>>>>>>>>> occasional updates
>>>>>>>>> every few month, it will never be restarted or need any
>>>>>>>>> manual
>>>>>>>>> maintenance
>>>>>>>>> at all. So I assume the code is not the badest on
>>>>>>>>> earth. At
>>>>>>>>> least
>>>>>>>>> because
>>>>>>>>> of that I'm able to out-rule things like memory leaks,
>>>>>>>>> gc
>>>>>>>>> issues
>>>>>>>>> or
>>>>>>>>> bottlenecks (max Heap is never reached, neither half of
>>>>>>>>> it,
>>>>>>>>> even
>>>>>>>>> after
>>>>>>>>> running for month).
>>>>>>>>>>> Of course there could be aspects of JSF which I may
>>>>>>>>>>> not
>>>>>>>>>>> have
>>>>>>>>>>> understood
>>>>>>>>> correctly until now. But because there are so many new
>>>>>>>>> things
>>>>>>>>> to
>>>>>>>>> a JSF
>>>>>>>>> beginner like me, I'm running out of ideas what this
>>>>>>>>> could
>>>>>>>>> be. It
>>>>>>>>> is
>>>>>>>>> something that worked well 'til TomEE 8.0.6 (Prod and
>>>>>>>>> Dev
>>>>>>>>> mode)
>>>>>>>>> and
>>>>>>>>> suddenly don't work anymore on 8.0.12 in Dev mode (but
>>>>>>>>> still
>>>>>>>>> works great in
>>>>>>>>> Prod mode). There are only minor versions in-between.
>>>>>>>>> So I
>>>>>>>>> think
>>>>>>>>> there were
>>>>>>>>> no revolutionary changes anywhere. My hope was (and
>>>>>>>>> still
>>>>>>>>> is),
>>>>>>>>> that anybody
>>>>>>>>> here on the list might have had a similar problem and
>>>>>>>>> has
>>>>>>>>> solved
>>>>>>>>> it, or
>>>>>>>>> maybe someone knows about some changes in Dev mode
>>>>>>>>> which
>>>>>>>>> could
>>>>>>>>> explain such
>>>>>>>>> behaviour (don't know, maybe xml parsing or component
>>>>>>>>> tree
>>>>>>>>> generation has
>>>>>>>>> changed, or scope handling of managed beans, or
>>>>>>>>> whatever
>>>>>>>>> could
>>>>>>>>> cause some
>>>>>>>>> code to be entered far more often than before, or some
>>>>>>>>> cache/pool
>>>>>>>>> was
>>>>>>>>> removed or has different behaviour... such things).
>>>>>>>>> Then it
>>>>>>>>> would
>>>>>>>>> be far
>>>>>>>>> more easy for me to track down the real problem.
>>>>>>>>>>> Anyway, when I get some spare time (currently have
>>>>>>>>>>> some
>>>>>>>>>>> deadlines in
>>>>>>>>> sight), I'll try to build a small test case. Either I
>>>>>>>>> can get
>>>>>>>>> a
>>>>>>>>> grip on the
>>>>>>>>> issue myself on doing this, or I can supply it to
>>>>>>>>> anyone who
>>>>>>>>> might want to
>>>>>>>>> have a look at it.
>>>>>>>>>>> cu
>>>>>>>>>>> Jens
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
>>>>>>>>>>>>> newer versions are so incredibly slow in dev
>>>>>>>>>>>>> mode
>>>>>>>>>>>> I can assure you that Dev/Prod mode works very
>>>>>>>>>>>> very
>>>>>>>>>>>> swiftly
>>>>>>>>>>>> on 8.0.15
>>>>>>>>>>>> for sizable applications. For example, I have a
>>>>>>>>>>>> giant
>>>>>>>>>>>> application
>>>>>>>>>>>> (thousands of LOC, 50+ jsf views) that has sub-
>>>>>>>>>>>> 8ms
>>>>>>>>>>>> response
>>>>>>>>>>>> times
>>>>>>>>>>>> (minus database time). More than likely they are
>>>>>>>>>>>> doing
>>>>>>>>>>>> something they
>>>>>>>>>>>> should not be doing with JSF apis and are causing
>>>>>>>>>>>> problems.
>>>>>>>>>>>>
>>>>>>>>>>>> I would check the obvious things first with
>>>>>>>>>>>> VisualVM:
>>>>>>>>>>>> heap
>>>>>>>>>>>> size, gc
>>>>>>>>>>>> frequency, etc. You can also run their profiler
>>>>>>>>>>>> and see
>>>>>>>>>>>> if
>>>>>>>>>>>> you can
>>>>>>>>>>>> spot anything immediately obvious.
>>>>>>>>>>>>
>>>>>>>>>>>> After that, the easiest way to locate the code
>>>>>>>>>>>> slowing
>>>>>>>>>>>> you
>>>>>>>>>>>> down is
>>>>>>>>>>>> bisection. Cut half the code out, run, and
>>>>>>>>>>>> continue
>>>>>>>>>>>> cutting
>>>>>>>>>>>> until you
>>>>>>>>>>>> can locate the problematic code. Then after that,
>>>>>>>>>>>> isolate a
>>>>>>>>>>>> test case
>>>>>>>>>>>> where flipping a boolean flag triggers the issue
>>>>>>>>>>>> and
>>>>>>>>>>>> post
>>>>>>>>>>>> the results
>>>>>>>>>>>> here.
>>>>>>>>>>>>
>>>>>>>>>>>> I would get ahold of:
>>>>>>>>>>>> https://www.tomitribe.com who are
>>>>>>>>>>>> literal
>>>>>>>>>>>> experts in this stuff. They offer professional
>>>>>>>>>>>> services
>>>>>>>>>>>> to
>>>>>>>>>>>> handle
>>>>>>>>>>>> exactly these situations and can work directly in
>>>>>>>>>>>> your
>>>>>>>>>>>> codebase to
>>>>>>>>>>>> help you find and fix the problem.
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski
>>>>>>>>>>>> <jz...@diegurus.de> wrote:
>>>>>>>>>>>>> Hi altogether,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm new to this list and hope it's the right
>>>>>>>>>>>>> place to
>>>>>>>>>>>>> ask
>>>>>>>>>>>>> this
>>>>>>>>> question.
>>>>>>>>>>>>> If not, and someone know the right place to ask
>>>>>>>>>>>>> this,
>>>>>>>>>>>>> please give
>>>>>>>>> advise.
>>>>>>>>>>>>> I'm developing a big JSF Application for a
>>>>>>>>>>>>> customer.
>>>>>>>>>>>>> It's
>>>>>>>>>>>>> a long
>>>>>>>>> running
>>>>>>>>>>>>> project and development and when I started with
>>>>>>>>>>>>> it,
>>>>>>>>>>>>> TomEE
>>>>>>>>>>>>> 8.0.6 was
>>>>>>>>> the
>>>>>>>>>>>>> most recent Version of TomEE. The switch to
>>>>>>>>>>>>> Jakarta
>>>>>>>>>>>>> is
>>>>>>>>>>>>> planned for
>>>>>>>>> next
>>>>>>>>>>>>> year, so I'm still on the 8.x path for the time
>>>>>>>>>>>>> being
>>>>>>>>>>>>> and
>>>>>>>>>>>>> wasn't able
>>>>>>>>> to
>>>>>>>>>>>>> test if this problem still exists in the 9.x
>>>>>>>>>>>>> branch.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The problem:
>>>>>>>>>>>>> On the customers site I'm with the latest TomEE
>>>>>>>>>>>>> version
>>>>>>>>>>>>> 8.0.15
>>>>>>>>> (running
>>>>>>>>>>>>> on Java 11) and everything works fine, because
>>>>>>>>>>>>> it is
>>>>>>>>>>>>> running in
>>>>>>>>>>>>> production mode. But in my development
>>>>>>>>>>>>> environment
>>>>>>>>>>>>> I'm
>>>>>>>>>>>>> still stuck
>>>>>>>>> with
>>>>>>>>>>>>> 8.0.6, because there I need the development
>>>>>>>>>>>>> mode
>>>>>>>>>>>>> (javax.faces.PROJECT_STAGE: Development). I
>>>>>>>>>>>>> need to
>>>>>>>>>>>>> be
>>>>>>>>>>>>> able to see
>>>>>>>>>>>>> changes in facelets without restarting
>>>>>>>>>>>>> everything
>>>>>>>>>>>>> everytime, and to
>>>>>>>>> get
>>>>>>>>>>>>> more detailed error messages.
>>>>>>>>>>>>> My attempts to update my dev TomEE to something
>>>>>>>>>>>>> newer
>>>>>>>>>>>>> than 8.0.6 all
>>>>>>>>>>>>> failed, because all (at least all I've tested
>>>>>>>>>>>>> so far)
>>>>>>>>>>>>> newer versions
>>>>>>>>> are
>>>>>>>>>>>>> so incredibly slow in dev mode, that it's
>>>>>>>>>>>>> unbearable
>>>>>>>>>>>>> to
>>>>>>>>>>>>> use the
>>>>>>>>>>>>> application. Several very looong seconds on
>>>>>>>>>>>>> every
>>>>>>>>>>>>> request
>>>>>>>>>>>>> (even the
>>>>>>>>>>>>> little AJAX requests in a view) is simply not a
>>>>>>>>>>>>> practical
>>>>>>>>>>>>> environment.
>>>>>>>>>>>>> After switching to Production mode everything
>>>>>>>>>>>>> works
>>>>>>>>>>>>> very
>>>>>>>>>>>>> fast, and all
>>>>>>>>>>>>> my views have response times of very few ms
>>>>>>>>>>>>> (even the
>>>>>>>>>>>>> big
>>>>>>>>>>>>> ones with
>>>>>>>>> max.
>>>>>>>>>>>>> around 400ms). When switching back to
>>>>>>>>>>>>> Development
>>>>>>>>>>>>> mode I
>>>>>>>>>>>>> have response
>>>>>>>>>>>>> times of up to 30s on big views. Not with
>>>>>>>>>>>>> 8.0.6,
>>>>>>>>>>>>> there
>>>>>>>>>>>>> even in
>>>>>>>>>>>>> development mode it's reasonable fast in not
>>>>>>>>>>>>> getting
>>>>>>>>>>>>> higher than 1s.
>>>>>>>>>>>>>
>>>>>>>>>>>>> My question:
>>>>>>>>>>>>> What causes this enormous performance
>>>>>>>>>>>>> degradation?
>>>>>>>>>>>>> I'm
>>>>>>>>>>>>> hoping, it is
>>>>>>>>>>>>> just a configuration which now has another
>>>>>>>>>>>>> default
>>>>>>>>>>>>> value
>>>>>>>>>>>>> or the like.
>>>>>>>>> If
>>>>>>>>>>>>> yes, maybe someone can point me in the right
>>>>>>>>>>>>> direction of
>>>>>>>>>>>>> where to
>>>>>>>>> find
>>>>>>>>>>>>> this configuration? If it's not a configuration
>>>>>>>>>>>>> thing:
>>>>>>>>>>>>> what can I do
>>>>>>>>> to
>>>>>>>>>>>>> get around this?
>>>>>>>>>>>>>
>>>>>>>>>>>>> I haven't tested all Versions of TomEE from
>>>>>>>>>>>>> 8.0.6 to
>>>>>>>>>>>>> 8.0.15, so I
>>>>>>>>> can't
>>>>>>>>>>>>> say at what version exactly this behaviour
>>>>>>>>>>>>> changes.
>>>>>>>>>>>>> If it
>>>>>>>>>>>>> helps or is
>>>>>>>>>>>>> needed, I can make some tests to find the exact
>>>>>>>>>>>>> version
>>>>>>>>>>>>> where this
>>>>>>>>>>>>> happens. Versions I've already tested are:
>>>>>>>>>>>>> 8.0.12,
>>>>>>>>>>>>> 8.0.13
>>>>>>>>>>>>> and 8.0.15.
>>>>>>>>>>>>> All of them are very slow in dev mode. If you
>>>>>>>>>>>>> need
>>>>>>>>>>>>> more
>>>>>>>>>>>>> information,
>>>>>>>>>>>>> I'll try to provide it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks in advance for any help
>>>>>>>>>>>>> cu
>>>>>>>>>>>>> Jens
>>>>>>>>>>>>>
>>>>>>>>>>>>>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Richard Zowalla <ri...@zowalla.com>.
Great news! (though I am not a Windows user anymore).
You should be able to build TomEE on Windows, too. If that does not
work out-of-the-box, you could try to build within a docker container
or via git-bash in Windows
(I am not 100% sure a Windows build will work out of the box)
I agree, that it might complicate things for testing purpose as it
increases the amount of cycles needed, but that are already very good
insights!
Gruß
Richard
Am Freitag, dem 27.10.2023 um 16:55 +0200 schrieb Jens Zurawski:
> I have a first result on this issue:
>
> The problem does not occur if TomEE is running on Linux. I've tested
> my
> own built 8.0.8, a freshly extracted 8.0.8 and 8.0.15 from the
> releases,
> and none of them showed the dev mode performance problem.
>
> Using TomEE 8.0.8 on Windows does show the dev mode performance
> problem.
> All of the tested ones, means the released 8.0.8 and my own built
> 8.0.8
> (the one I've built on Linux).
>
> My development environment is Windows 10 (Netbeans and therefore also
> the TomEE for easy deployment and debugging). The problem does occur
> on
> stand alone started TomEE and also if started out of Netbeans, so
> Netbeans can be sorted out as the root of the problem. Java Versions
> 11,
> 13, 15 and 17 tested, all the same, so Java should also not be the
> problem. BTW: It is very unlikely a problem of my individual Windows
> installation, because this problem also shows up on the Windows
> system
> of my colleague, which btw. was the root cause why I'm started to dig
> deeper into this problem now and why I subscribed to this mailing
> list ;-)
>
> I'm afraid, this will complicate my git bisect session a little bit
> :-(
>
> However, I'll keep you posted.
>
> cu
> Jens
>
>
> Am 27.10.2023 um 13:12 schrieb Jens Zurawski:
> > Thanks Richard (and Vicente),
> >
> > I've changed to JDK 8 and now the builds are successful (both 8.0.6
> > and 8.0.8).
> >
> > I'll script me some test scripts to ease the build, run and test
> > for
> > every bisect step.
> > A build needs around 3 to 4 minutes. Running a test with my
> > application will need some manual interaction and checks, I guess
> > also
> > something around 4 minutes. This sums up to roughly 8 minutes per
> > step.
> >
> > I've started a bisect and it told me:
> > git bisect good 20441eb
> > git bisect bad 4c8a616
> > Bisecting: 136 revisions left to test after this (roughly 7 steps)
> >
> > That means, if everything works fine, I should be able to identify
> > the
> > commit after about an hour? (That's cool.... why nobody has told me
> > about git bisect before? ;-) )
> >
> > Anyway, I will have to find some time for this, maybe I need a
> > couple
> > of days to find some.
> > I will let you know about the outcome.
> >
> > cu
> > Jens
> >
> >
> > Am 27.10.2023 um 11:50 schrieb Richard Zowalla:
> > > Hi,
> > >
> > > you need Java/JDK 8 to build TomEE 8.
> > >
> > > The Maven command is
> > >
> > > mvn --fail-at-end clean install -pl tomee/apache-tomee -am -
> > > Dfile.encoding=UTF-8 -DskipTests=true
> > >
> > > You will find the tar.gz/zip in ./tomee/apache-tomee/target
> > >
> > > Gruß
> > > Richard
> > >
> > > P.S. 8.0.7 is here:
> > > https://archive.apache.org/dist/tomee/tomee-8.0.7/
> > >
> > > Am Freitag, dem 27.10.2023 um 10:50 +0200 schrieb Jens Zurawski:
> > > > Hi Richard,
> > > >
> > > > to identify the "bad" version was easy. I didn't find a 8.0.7
> > > > release
> > > > on
> > > > the download page, so I started with 8.0.8. And this version
> > > > already
> > > > has
> > > > the slow dev mode. So for my problem I can say:
> > > > good: 8.0.6
> > > > bad: 8.0.8
> > > >
> > > > Now, I think, I need a little help from you. I tried to build a
> > > > TomEE
> > > > out of git (I've cloned the github repository
> > > > https://github.com/apache/tomee.git). I've tried these two
> > > > commits:
> > > > 20441eb (tag: tomee-8.0.6)
> > > > 4c8a616 (tag: tomee-project-8.0.8)
> > > >
> > > > But the build fails every time on "TomEE :: Container :: Core"
> > > > with:
> > > >
> > > > [INFO] BUILD FAILURE
> > > > [INFO]
> > > > ---------------------------------------------------------------
> > > > ------
> > > > ---
> > > > [INFO] Total time: 01:19 min
> > > > [INFO] Finished at: 2023-10-27T10:29:40+02:00
> > > > [INFO]
> > > > ---------------------------------------------------------------
> > > > ------
> > > > ---
> > > > [ERROR] Failed to execute goal
> > > > org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile
> > > > (default-compile) on project openejb-core: Compilation failure:
> > > > Compilation failure:
> > > > [ERROR]
> > > > /sandbox/tomee/container/openejb-
> > > > core/src/main/java/org/apache/openejb/core/OrbFactory.java:[21,
> > > > 21]
> > > > package org.omg.CORBA does not exist
> > > > [ERROR]
> > > > /sandbox/tomee/container/openejb-
> > > > core/src/main/java/org/apache/openejb/core/OrbFactory.java:[24,
> > > > 12]
> > > > cannot find symbol
> > > > [ERROR] symbol: class ORB
> > > > [ERROR] location: class org.apache.openejb.core.OrbFactory
> > > >
> > > > I've tried to build with:
> > > > mvn clean install
> > > >
> > > > My build environment is a linux ubuntu 20.04.6 LTS VM with:
> > > > git 2.25.1
> > > > Maven 3.6.3
> > > > java 14.0.2
> > > >
> > > > Is it some easy thing to fix, or do I have to switch some
> > > > versions of
> > > > my
> > > > environment?
> > > >
> > > > cu
> > > > Jens
> > > >
> > > > Am 26.10.2023 um 19:48 schrieb Richard Zowalla:
> > > > > Hi,
> > > > >
> > > > > without knowing code or having a reproducer, it might be not
> > > > > that
> > > > > obvious to find a reason / answer for the "academic problem"
> > > > > ;-)
> > > > >
> > > > > If you really want to boil it down, I recommend the following
> > > > > approach:
> > > > >
> > > > > (1) From your description it seems, that 8.0.6 is the last
> > > > > known
> > > > > "good"
> > > > > version.
> > > > >
> > > > > (2) You've investigated versions starting from 8.0.12
> > > > > onwards,
> > > > > which
> > > > > seems to be "bad".
> > > > >
> > > > > Following the approach recommended by Jonathan:
> > > > >
> > > > > - Can you check if it happens in 8.0.7 (8.0.8, ...) until we
> > > > > find
> > > > > the
> > > > > first "bad" released version?
> > > > >
> > > > > - If we have that, you could start a "git bisect" session
> > > > > between
> > > > > 8.0.6
> > > > > and the first version, which is known to be bad (reducing the
> > > > > amount of
> > > > > needed bisect cycles dramatically), so we find a commit /
> > > > > reason
> > > > > (if it
> > > > > is in TomEE).
> > > > >
> > > > > - It would involve some quick builds without executing tests
> > > > > (~10-
> > > > > 15min
> > > > > each) for each bisect step and testing on your side.
> > > > >
> > > > > Don't know if your inner monk is willing to go that route,
> > > > > but I am
> > > > > happy to help with the related Maven commands for quick
> > > > > building :)
> > > > >
> > > > > Gruß
> > > > > Richard
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Am Donnerstag, dem 26.10.2023 um 18:00 +0200 schrieb Jens
> > > > > Zurawski:
> > > > > > Hi Vicente,
> > > > > >
> > > > > > thank you for the tip. This is a good workaround. When I
> > > > > > switch
> > > > > > to
> > > > > > Production mode and insert this configuration, it reloads
> > > > > > the
> > > > > > view
> > > > > > from
> > > > > > the file if it has changed. This means, I can use a decent
> > > > > > version of
> > > > > > TomEE in my dev environment.
> > > > > >
> > > > > > However, it doesn't completely solve my problem, as the
> > > > > > error
> > > > > > reporting
> > > > > > isn't as verbose as in Dev mode as far as I can see. But I
> > > > > > have
> > > > > > to
> > > > > > check, maybe it's enough for my development.
> > > > > >
> > > > > > And, well.... the academic problem: "Why is it still broken
> > > > > > in
> > > > > > Dev
> > > > > > mode?" still runs circles through my brain ;-) Because if
> > > > > > it's a
> > > > > > problem
> > > > > > of my coding, it might be a good idea to change this,
> > > > > > before it
> > > > > > might
> > > > > > get a problem on Production mode, too, at some time in the
> > > > > > future.
> > > > > >
> > > > > > But anyway, you've helped me a good step forward, thanks.
> > > > > > cu
> > > > > > Jens
> > > > > >
> > > > > >
> > > > > > Am 26.10.2023 um 17:04 schrieb Vicente Rossello:
> > > > > > > We are using the production stage with
> > > > > > > javax.faces.FACELETS_REFRESH_PERIOD
> > > > > > > = 1. It's faster and more similar to production, just try
> > > > > > > it if
> > > > > > > it
> > > > > > > solves
> > > > > > > your problem
> > > > > > >
> > > > > > > On Thu, Oct 26, 2023 at 4:18 PM Jens Zurawski
> > > > > > > <jz...@diegurus.de>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > The one bundled with TomEE.
> > > > > > > > myfaces 2.3.7 on TomEE 8.0.6
> > > > > > > > and e.g. myfaces 2.3.9 on TomEE 8.0.12
> > > > > > > >
> > > > > > > > Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
> > > > > > > > > Which JSF lib are you using?
> > > > > > > > >
> > > > > > > > > Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens
> > > > > > > > > Zurawski
> > > > > > > > > <jzu@diegurus.de
> > > > > > > > > :
> > > > > > > > > > Thank you for your quick reply, Jonathan.
> > > > > > > > > >
> > > > > > > > > > It is good to know, that normally the Dev mode
> > > > > > > > > > should
> > > > > > > > > > work as
> > > > > > > > > > fast as
> > > > > > > > before even with 8.0.15. So it obviously has something
> > > > > > > > to do
> > > > > > > > with
> > > > > > > > my
> > > > > > > > environment, configuration and/or code.
> > > > > > > > > > And also thank you for the reference to tomitribe.
> > > > > > > > > > Unfortunately I'm
> > > > > > > > the only developer on this project, and it will be
> > > > > > > > difficult
> > > > > > > > to
> > > > > > > > convince my
> > > > > > > > customer to an additional support subscription, because
> > > > > > > > he on
> > > > > > > > production
> > > > > > > > doesn't have issues ;-) But, ok, that's my problem, and
> > > > > > > > maybe
> > > > > > > > I'll consult
> > > > > > > > tomitribe in the future for other projects.
> > > > > > > > > > So, I'm afraid, for the near future I'm on my own
> > > > > > > > > > to
> > > > > > > > > > solve
> > > > > > > > > > this
> > > > > > > > problem. Therefore any hints to restrict the places
> > > > > > > > where I
> > > > > > > > should take a
> > > > > > > > look will be more than welcome.
> > > > > > > > > > I'm a senior programmer with 20 years of experience
> > > > > > > > > > in
> > > > > > > > > > Java
> > > > > > > > > > programming
> > > > > > > > (SE and JSP). But I'm relative new to
> > > > > > > > JSF/JavaEE/JakartaEE
> > > > > > > > and
> > > > > > > > sometimes I
> > > > > > > > have a hard time to understand which component of the
> > > > > > > > whole
> > > > > > > > stack
> > > > > > > > is
> > > > > > > > responsible for what.
> > > > > > > > > > The project meanwhile has over a 1/4 million lines
> > > > > > > > > > of
> > > > > > > > > > code
> > > > > > > > > > (most of
> > > > > > > > them in the back-end, but also the front-end is big
> > > > > > > > enough to
> > > > > > > > not
> > > > > > > > trace
> > > > > > > > through every lane if I want to get to the point in
> > > > > > > > this life
> > > > > > > > :-D
> > > > > > > > ).
> > > > > > > > > > The application now runs for nearly 2 years on the
> > > > > > > > > > customers
> > > > > > > > > > site and
> > > > > > > > is very reliable and fast in its daily usage. Apart
> > > > > > > > from
> > > > > > > > occasional updates
> > > > > > > > every few month, it will never be restarted or need any
> > > > > > > > manual
> > > > > > > > maintenance
> > > > > > > > at all. So I assume the code is not the badest on
> > > > > > > > earth. At
> > > > > > > > least
> > > > > > > > because
> > > > > > > > of that I'm able to out-rule things like memory leaks,
> > > > > > > > gc
> > > > > > > > issues
> > > > > > > > or
> > > > > > > > bottlenecks (max Heap is never reached, neither half of
> > > > > > > > it,
> > > > > > > > even
> > > > > > > > after
> > > > > > > > running for month).
> > > > > > > > > > Of course there could be aspects of JSF which I may
> > > > > > > > > > not
> > > > > > > > > > have
> > > > > > > > > > understood
> > > > > > > > correctly until now. But because there are so many new
> > > > > > > > things
> > > > > > > > to
> > > > > > > > a JSF
> > > > > > > > beginner like me, I'm running out of ideas what this
> > > > > > > > could
> > > > > > > > be. It
> > > > > > > > is
> > > > > > > > something that worked well 'til TomEE 8.0.6 (Prod and
> > > > > > > > Dev
> > > > > > > > mode)
> > > > > > > > and
> > > > > > > > suddenly don't work anymore on 8.0.12 in Dev mode (but
> > > > > > > > still
> > > > > > > > works great in
> > > > > > > > Prod mode). There are only minor versions in-between.
> > > > > > > > So I
> > > > > > > > think
> > > > > > > > there were
> > > > > > > > no revolutionary changes anywhere. My hope was (and
> > > > > > > > still
> > > > > > > > is),
> > > > > > > > that anybody
> > > > > > > > here on the list might have had a similar problem and
> > > > > > > > has
> > > > > > > > solved
> > > > > > > > it, or
> > > > > > > > maybe someone knows about some changes in Dev mode
> > > > > > > > which
> > > > > > > > could
> > > > > > > > explain such
> > > > > > > > behaviour (don't know, maybe xml parsing or component
> > > > > > > > tree
> > > > > > > > generation has
> > > > > > > > changed, or scope handling of managed beans, or
> > > > > > > > whatever
> > > > > > > > could
> > > > > > > > cause some
> > > > > > > > code to be entered far more often than before, or some
> > > > > > > > cache/pool
> > > > > > > > was
> > > > > > > > removed or has different behaviour... such things).
> > > > > > > > Then it
> > > > > > > > would
> > > > > > > > be far
> > > > > > > > more easy for me to track down the real problem.
> > > > > > > > > > Anyway, when I get some spare time (currently have
> > > > > > > > > > some
> > > > > > > > > > deadlines in
> > > > > > > > sight), I'll try to build a small test case. Either I
> > > > > > > > can get
> > > > > > > > a
> > > > > > > > grip on the
> > > > > > > > issue myself on doing this, or I can supply it to
> > > > > > > > anyone who
> > > > > > > > might want to
> > > > > > > > have a look at it.
> > > > > > > > > > cu
> > > > > > > > > > Jens
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
> > > > > > > > > > > > newer versions are so incredibly slow in dev
> > > > > > > > > > > > mode
> > > > > > > > > > > I can assure you that Dev/Prod mode works very
> > > > > > > > > > > very
> > > > > > > > > > > swiftly
> > > > > > > > > > > on 8.0.15
> > > > > > > > > > > for sizable applications. For example, I have a
> > > > > > > > > > > giant
> > > > > > > > > > > application
> > > > > > > > > > > (thousands of LOC, 50+ jsf views) that has sub-
> > > > > > > > > > > 8ms
> > > > > > > > > > > response
> > > > > > > > > > > times
> > > > > > > > > > > (minus database time). More than likely they are
> > > > > > > > > > > doing
> > > > > > > > > > > something they
> > > > > > > > > > > should not be doing with JSF apis and are causing
> > > > > > > > > > > problems.
> > > > > > > > > > >
> > > > > > > > > > > I would check the obvious things first with
> > > > > > > > > > > VisualVM:
> > > > > > > > > > > heap
> > > > > > > > > > > size, gc
> > > > > > > > > > > frequency, etc. You can also run their profiler
> > > > > > > > > > > and see
> > > > > > > > > > > if
> > > > > > > > > > > you can
> > > > > > > > > > > spot anything immediately obvious.
> > > > > > > > > > >
> > > > > > > > > > > After that, the easiest way to locate the code
> > > > > > > > > > > slowing
> > > > > > > > > > > you
> > > > > > > > > > > down is
> > > > > > > > > > > bisection. Cut half the code out, run, and
> > > > > > > > > > > continue
> > > > > > > > > > > cutting
> > > > > > > > > > > until you
> > > > > > > > > > > can locate the problematic code. Then after that,
> > > > > > > > > > > isolate a
> > > > > > > > > > > test case
> > > > > > > > > > > where flipping a boolean flag triggers the issue
> > > > > > > > > > > and
> > > > > > > > > > > post
> > > > > > > > > > > the results
> > > > > > > > > > > here.
> > > > > > > > > > >
> > > > > > > > > > > I would get ahold of:
> > > > > > > > > > > https://www.tomitribe.com who are
> > > > > > > > > > > literal
> > > > > > > > > > > experts in this stuff. They offer professional
> > > > > > > > > > > services
> > > > > > > > > > > to
> > > > > > > > > > > handle
> > > > > > > > > > > exactly these situations and can work directly in
> > > > > > > > > > > your
> > > > > > > > > > > codebase to
> > > > > > > > > > > help you find and fix the problem.
> > > > > > > > > > >
> > > > > > > > > > > On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski
> > > > > > > > > > > <jz...@diegurus.de> wrote:
> > > > > > > > > > > > Hi altogether,
> > > > > > > > > > > >
> > > > > > > > > > > > I'm new to this list and hope it's the right
> > > > > > > > > > > > place to
> > > > > > > > > > > > ask
> > > > > > > > > > > > this
> > > > > > > > question.
> > > > > > > > > > > > If not, and someone know the right place to ask
> > > > > > > > > > > > this,
> > > > > > > > > > > > please give
> > > > > > > > advise.
> > > > > > > > > > > > I'm developing a big JSF Application for a
> > > > > > > > > > > > customer.
> > > > > > > > > > > > It's
> > > > > > > > > > > > a long
> > > > > > > > running
> > > > > > > > > > > > project and development and when I started with
> > > > > > > > > > > > it,
> > > > > > > > > > > > TomEE
> > > > > > > > > > > > 8.0.6 was
> > > > > > > > the
> > > > > > > > > > > > most recent Version of TomEE. The switch to
> > > > > > > > > > > > Jakarta
> > > > > > > > > > > > is
> > > > > > > > > > > > planned for
> > > > > > > > next
> > > > > > > > > > > > year, so I'm still on the 8.x path for the time
> > > > > > > > > > > > being
> > > > > > > > > > > > and
> > > > > > > > > > > > wasn't able
> > > > > > > > to
> > > > > > > > > > > > test if this problem still exists in the 9.x
> > > > > > > > > > > > branch.
> > > > > > > > > > > >
> > > > > > > > > > > > The problem:
> > > > > > > > > > > > On the customers site I'm with the latest TomEE
> > > > > > > > > > > > version
> > > > > > > > > > > > 8.0.15
> > > > > > > > (running
> > > > > > > > > > > > on Java 11) and everything works fine, because
> > > > > > > > > > > > it is
> > > > > > > > > > > > running in
> > > > > > > > > > > > production mode. But in my development
> > > > > > > > > > > > environment
> > > > > > > > > > > > I'm
> > > > > > > > > > > > still stuck
> > > > > > > > with
> > > > > > > > > > > > 8.0.6, because there I need the development
> > > > > > > > > > > > mode
> > > > > > > > > > > > (javax.faces.PROJECT_STAGE: Development). I
> > > > > > > > > > > > need to
> > > > > > > > > > > > be
> > > > > > > > > > > > able to see
> > > > > > > > > > > > changes in facelets without restarting
> > > > > > > > > > > > everything
> > > > > > > > > > > > everytime, and to
> > > > > > > > get
> > > > > > > > > > > > more detailed error messages.
> > > > > > > > > > > > My attempts to update my dev TomEE to something
> > > > > > > > > > > > newer
> > > > > > > > > > > > than 8.0.6 all
> > > > > > > > > > > > failed, because all (at least all I've tested
> > > > > > > > > > > > so far)
> > > > > > > > > > > > newer versions
> > > > > > > > are
> > > > > > > > > > > > so incredibly slow in dev mode, that it's
> > > > > > > > > > > > unbearable
> > > > > > > > > > > > to
> > > > > > > > > > > > use the
> > > > > > > > > > > > application. Several very looong seconds on
> > > > > > > > > > > > every
> > > > > > > > > > > > request
> > > > > > > > > > > > (even the
> > > > > > > > > > > > little AJAX requests in a view) is simply not a
> > > > > > > > > > > > practical
> > > > > > > > > > > > environment.
> > > > > > > > > > > > After switching to Production mode everything
> > > > > > > > > > > > works
> > > > > > > > > > > > very
> > > > > > > > > > > > fast, and all
> > > > > > > > > > > > my views have response times of very few ms
> > > > > > > > > > > > (even the
> > > > > > > > > > > > big
> > > > > > > > > > > > ones with
> > > > > > > > max.
> > > > > > > > > > > > around 400ms). When switching back to
> > > > > > > > > > > > Development
> > > > > > > > > > > > mode I
> > > > > > > > > > > > have response
> > > > > > > > > > > > times of up to 30s on big views. Not with
> > > > > > > > > > > > 8.0.6,
> > > > > > > > > > > > there
> > > > > > > > > > > > even in
> > > > > > > > > > > > development mode it's reasonable fast in not
> > > > > > > > > > > > getting
> > > > > > > > > > > > higher than 1s.
> > > > > > > > > > > >
> > > > > > > > > > > > My question:
> > > > > > > > > > > > What causes this enormous performance
> > > > > > > > > > > > degradation?
> > > > > > > > > > > > I'm
> > > > > > > > > > > > hoping, it is
> > > > > > > > > > > > just a configuration which now has another
> > > > > > > > > > > > default
> > > > > > > > > > > > value
> > > > > > > > > > > > or the like.
> > > > > > > > If
> > > > > > > > > > > > yes, maybe someone can point me in the right
> > > > > > > > > > > > direction of
> > > > > > > > > > > > where to
> > > > > > > > find
> > > > > > > > > > > > this configuration? If it's not a configuration
> > > > > > > > > > > > thing:
> > > > > > > > > > > > what can I do
> > > > > > > > to
> > > > > > > > > > > > get around this?
> > > > > > > > > > > >
> > > > > > > > > > > > I haven't tested all Versions of TomEE from
> > > > > > > > > > > > 8.0.6 to
> > > > > > > > > > > > 8.0.15, so I
> > > > > > > > can't
> > > > > > > > > > > > say at what version exactly this behaviour
> > > > > > > > > > > > changes.
> > > > > > > > > > > > If it
> > > > > > > > > > > > helps or is
> > > > > > > > > > > > needed, I can make some tests to find the exact
> > > > > > > > > > > > version
> > > > > > > > > > > > where this
> > > > > > > > > > > > happens. Versions I've already tested are:
> > > > > > > > > > > > 8.0.12,
> > > > > > > > > > > > 8.0.13
> > > > > > > > > > > > and 8.0.15.
> > > > > > > > > > > > All of them are very slow in dev mode. If you
> > > > > > > > > > > > need
> > > > > > > > > > > > more
> > > > > > > > > > > > information,
> > > > > > > > > > > > I'll try to provide it.
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks in advance for any help
> > > > > > > > > > > > cu
> > > > > > > > > > > > Jens
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > >
>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Jens Zurawski <jz...@diegurus.de>.
Oh, wow, yes this did it. After using these switches, the response times
in dev mode are getting down to the values I'm used to have with TomEE
8.0.6.
Thank you very much.
I surely will not use this in production, my production targets are
solely linux system, because I refuse to admin/manage Windows Servers ;-)
About what kind of "dirty trick" are you writing? If it helps to myke my
dev life easier without "corrupting" the production environments, I'm
interested.
cu
Jens
Am 27.10.2023 um 17:08 schrieb Vicente Rossello:
> I want through this, now I remember. Try with these system switches
>
> -Dsun.io.useCanonCaches=true -Dsun.io.useCanonPrefixCache=true
>
> Dont use this in production, be aware.
>
> We ended up removing these flash and using a dirty trick with tomcat
> DirResourcesSet
>
> I can send you this tonight. Try the flags to see if we are talking about
> the same problem
>
>
> El vie, 27 oct 2023, 16:56, Jens Zurawski <jz...@diegurus.de> escribió:
>
>> I have a first result on this issue:
>>
>> The problem does not occur if TomEE is running on Linux. I've tested my
>> own built 8.0.8, a freshly extracted 8.0.8 and 8.0.15 from the releases,
>> and none of them showed the dev mode performance problem.
>>
>> Using TomEE 8.0.8 on Windows does show the dev mode performance problem.
>> All of the tested ones, means the released 8.0.8 and my own built 8.0.8
>> (the one I've built on Linux).
>>
>> My development environment is Windows 10 (Netbeans and therefore also
>> the TomEE for easy deployment and debugging). The problem does occur on
>> stand alone started TomEE and also if started out of Netbeans, so
>> Netbeans can be sorted out as the root of the problem. Java Versions 11,
>> 13, 15 and 17 tested, all the same, so Java should also not be the
>> problem. BTW: It is very unlikely a problem of my individual Windows
>> installation, because this problem also shows up on the Windows system
>> of my colleague, which btw. was the root cause why I'm started to dig
>> deeper into this problem now and why I subscribed to this mailing list ;-)
>>
>> I'm afraid, this will complicate my git bisect session a little bit :-(
>>
>> However, I'll keep you posted.
>>
>> cu
>> Jens
>>
>>
>> Am 27.10.2023 um 13:12 schrieb Jens Zurawski:
>>> Thanks Richard (and Vicente),
>>>
>>> I've changed to JDK 8 and now the builds are successful (both 8.0.6
>>> and 8.0.8).
>>>
>>> I'll script me some test scripts to ease the build, run and test for
>>> every bisect step.
>>> A build needs around 3 to 4 minutes. Running a test with my
>>> application will need some manual interaction and checks, I guess also
>>> something around 4 minutes. This sums up to roughly 8 minutes per step.
>>>
>>> I've started a bisect and it told me:
>>> git bisect good 20441eb
>>> git bisect bad 4c8a616
>>> Bisecting: 136 revisions left to test after this (roughly 7 steps)
>>>
>>> That means, if everything works fine, I should be able to identify the
>>> commit after about an hour? (That's cool.... why nobody has told me
>>> about git bisect before? ;-) )
>>>
>>> Anyway, I will have to find some time for this, maybe I need a couple
>>> of days to find some.
>>> I will let you know about the outcome.
>>>
>>> cu
>>> Jens
>>>
>>>
>>> Am 27.10.2023 um 11:50 schrieb Richard Zowalla:
>>>> Hi,
>>>>
>>>> you need Java/JDK 8 to build TomEE 8.
>>>>
>>>> The Maven command is
>>>>
>>>> mvn --fail-at-end clean install -pl tomee/apache-tomee -am -
>>>> Dfile.encoding=UTF-8 -DskipTests=true
>>>>
>>>> You will find the tar.gz/zip in ./tomee/apache-tomee/target
>>>>
>>>> Gruß
>>>> Richard
>>>>
>>>> P.S. 8.0.7 is here: https://archive.apache.org/dist/tomee/tomee-8.0.7/
>>>>
>>>> Am Freitag, dem 27.10.2023 um 10:50 +0200 schrieb Jens Zurawski:
>>>>> Hi Richard,
>>>>>
>>>>> to identify the "bad" version was easy. I didn't find a 8.0.7 release
>>>>> on
>>>>> the download page, so I started with 8.0.8. And this version already
>>>>> has
>>>>> the slow dev mode. So for my problem I can say:
>>>>> good: 8.0.6
>>>>> bad: 8.0.8
>>>>>
>>>>> Now, I think, I need a little help from you. I tried to build a TomEE
>>>>> out of git (I've cloned the github repository
>>>>> https://github.com/apache/tomee.git). I've tried these two commits:
>>>>> 20441eb (tag: tomee-8.0.6)
>>>>> 4c8a616 (tag: tomee-project-8.0.8)
>>>>>
>>>>> But the build fails every time on "TomEE :: Container :: Core" with:
>>>>>
>>>>> [INFO] BUILD FAILURE
>>>>> [INFO]
>>>>> ---------------------------------------------------------------------
>>>>> ---
>>>>> [INFO] Total time: 01:19 min
>>>>> [INFO] Finished at: 2023-10-27T10:29:40+02:00
>>>>> [INFO]
>>>>> ---------------------------------------------------------------------
>>>>> ---
>>>>> [ERROR] Failed to execute goal
>>>>> org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile
>>>>> (default-compile) on project openejb-core: Compilation failure:
>>>>> Compilation failure:
>>>>> [ERROR]
>>>>> /sandbox/tomee/container/openejb-
>>>>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[21,21]
>>>>> package org.omg.CORBA does not exist
>>>>> [ERROR]
>>>>> /sandbox/tomee/container/openejb-
>>>>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[24,12]
>>>>> cannot find symbol
>>>>> [ERROR] symbol: class ORB
>>>>> [ERROR] location: class org.apache.openejb.core.OrbFactory
>>>>>
>>>>> I've tried to build with:
>>>>> mvn clean install
>>>>>
>>>>> My build environment is a linux ubuntu 20.04.6 LTS VM with:
>>>>> git 2.25.1
>>>>> Maven 3.6.3
>>>>> java 14.0.2
>>>>>
>>>>> Is it some easy thing to fix, or do I have to switch some versions of
>>>>> my
>>>>> environment?
>>>>>
>>>>> cu
>>>>> Jens
>>>>>
>>>>> Am 26.10.2023 um 19:48 schrieb Richard Zowalla:
>>>>>> Hi,
>>>>>>
>>>>>> without knowing code or having a reproducer, it might be not that
>>>>>> obvious to find a reason / answer for the "academic problem" ;-)
>>>>>>
>>>>>> If you really want to boil it down, I recommend the following
>>>>>> approach:
>>>>>>
>>>>>> (1) From your description it seems, that 8.0.6 is the last known
>>>>>> "good"
>>>>>> version.
>>>>>>
>>>>>> (2) You've investigated versions starting from 8.0.12 onwards,
>>>>>> which
>>>>>> seems to be "bad".
>>>>>>
>>>>>> Following the approach recommended by Jonathan:
>>>>>>
>>>>>> - Can you check if it happens in 8.0.7 (8.0.8, ...) until we find
>>>>>> the
>>>>>> first "bad" released version?
>>>>>>
>>>>>> - If we have that, you could start a "git bisect" session between
>>>>>> 8.0.6
>>>>>> and the first version, which is known to be bad (reducing the
>>>>>> amount of
>>>>>> needed bisect cycles dramatically), so we find a commit / reason
>>>>>> (if it
>>>>>> is in TomEE).
>>>>>>
>>>>>> - It would involve some quick builds without executing tests (~10-
>>>>>> 15min
>>>>>> each) for each bisect step and testing on your side.
>>>>>>
>>>>>> Don't know if your inner monk is willing to go that route, but I am
>>>>>> happy to help with the related Maven commands for quick building :)
>>>>>>
>>>>>> Gruß
>>>>>> Richard
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Am Donnerstag, dem 26.10.2023 um 18:00 +0200 schrieb Jens Zurawski:
>>>>>>> Hi Vicente,
>>>>>>>
>>>>>>> thank you for the tip. This is a good workaround. When I switch
>>>>>>> to
>>>>>>> Production mode and insert this configuration, it reloads the
>>>>>>> view
>>>>>>> from
>>>>>>> the file if it has changed. This means, I can use a decent
>>>>>>> version of
>>>>>>> TomEE in my dev environment.
>>>>>>>
>>>>>>> However, it doesn't completely solve my problem, as the error
>>>>>>> reporting
>>>>>>> isn't as verbose as in Dev mode as far as I can see. But I have
>>>>>>> to
>>>>>>> check, maybe it's enough for my development.
>>>>>>>
>>>>>>> And, well.... the academic problem: "Why is it still broken in
>>>>>>> Dev
>>>>>>> mode?" still runs circles through my brain ;-) Because if it's a
>>>>>>> problem
>>>>>>> of my coding, it might be a good idea to change this, before it
>>>>>>> might
>>>>>>> get a problem on Production mode, too, at some time in the
>>>>>>> future.
>>>>>>>
>>>>>>> But anyway, you've helped me a good step forward, thanks.
>>>>>>> cu
>>>>>>> Jens
>>>>>>>
>>>>>>>
>>>>>>> Am 26.10.2023 um 17:04 schrieb Vicente Rossello:
>>>>>>>> We are using the production stage with
>>>>>>>> javax.faces.FACELETS_REFRESH_PERIOD
>>>>>>>> = 1. It's faster and more similar to production, just try it if
>>>>>>>> it
>>>>>>>> solves
>>>>>>>> your problem
>>>>>>>>
>>>>>>>> On Thu, Oct 26, 2023 at 4:18 PM Jens Zurawski <jz...@diegurus.de>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> The one bundled with TomEE.
>>>>>>>>> myfaces 2.3.7 on TomEE 8.0.6
>>>>>>>>> and e.g. myfaces 2.3.9 on TomEE 8.0.12
>>>>>>>>>
>>>>>>>>> Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
>>>>>>>>>> Which JSF lib are you using?
>>>>>>>>>>
>>>>>>>>>> Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens Zurawski
>>>>>>>>>> <jzu@diegurus.de
>>>>>>>>>> :
>>>>>>>>>>> Thank you for your quick reply, Jonathan.
>>>>>>>>>>>
>>>>>>>>>>> It is good to know, that normally the Dev mode should
>>>>>>>>>>> work as
>>>>>>>>>>> fast as
>>>>>>>>> before even with 8.0.15. So it obviously has something to do
>>>>>>>>> with
>>>>>>>>> my
>>>>>>>>> environment, configuration and/or code.
>>>>>>>>>>> And also thank you for the reference to tomitribe.
>>>>>>>>>>> Unfortunately I'm
>>>>>>>>> the only developer on this project, and it will be difficult
>>>>>>>>> to
>>>>>>>>> convince my
>>>>>>>>> customer to an additional support subscription, because he on
>>>>>>>>> production
>>>>>>>>> doesn't have issues ;-) But, ok, that's my problem, and maybe
>>>>>>>>> I'll consult
>>>>>>>>> tomitribe in the future for other projects.
>>>>>>>>>>> So, I'm afraid, for the near future I'm on my own to
>>>>>>>>>>> solve
>>>>>>>>>>> this
>>>>>>>>> problem. Therefore any hints to restrict the places where I
>>>>>>>>> should take a
>>>>>>>>> look will be more than welcome.
>>>>>>>>>>> I'm a senior programmer with 20 years of experience in
>>>>>>>>>>> Java
>>>>>>>>>>> programming
>>>>>>>>> (SE and JSP). But I'm relative new to JSF/JavaEE/JakartaEE
>>>>>>>>> and
>>>>>>>>> sometimes I
>>>>>>>>> have a hard time to understand which component of the whole
>>>>>>>>> stack
>>>>>>>>> is
>>>>>>>>> responsible for what.
>>>>>>>>>>> The project meanwhile has over a 1/4 million lines of
>>>>>>>>>>> code
>>>>>>>>>>> (most of
>>>>>>>>> them in the back-end, but also the front-end is big enough to
>>>>>>>>> not
>>>>>>>>> trace
>>>>>>>>> through every lane if I want to get to the point in this life
>>>>>>>>> :-D
>>>>>>>>> ).
>>>>>>>>>>> The application now runs for nearly 2 years on the
>>>>>>>>>>> customers
>>>>>>>>>>> site and
>>>>>>>>> is very reliable and fast in its daily usage. Apart from
>>>>>>>>> occasional updates
>>>>>>>>> every few month, it will never be restarted or need any
>>>>>>>>> manual
>>>>>>>>> maintenance
>>>>>>>>> at all. So I assume the code is not the badest on earth. At
>>>>>>>>> least
>>>>>>>>> because
>>>>>>>>> of that I'm able to out-rule things like memory leaks, gc
>>>>>>>>> issues
>>>>>>>>> or
>>>>>>>>> bottlenecks (max Heap is never reached, neither half of it,
>>>>>>>>> even
>>>>>>>>> after
>>>>>>>>> running for month).
>>>>>>>>>>> Of course there could be aspects of JSF which I may not
>>>>>>>>>>> have
>>>>>>>>>>> understood
>>>>>>>>> correctly until now. But because there are so many new things
>>>>>>>>> to
>>>>>>>>> a JSF
>>>>>>>>> beginner like me, I'm running out of ideas what this could
>>>>>>>>> be. It
>>>>>>>>> is
>>>>>>>>> something that worked well 'til TomEE 8.0.6 (Prod and Dev
>>>>>>>>> mode)
>>>>>>>>> and
>>>>>>>>> suddenly don't work anymore on 8.0.12 in Dev mode (but still
>>>>>>>>> works great in
>>>>>>>>> Prod mode). There are only minor versions in-between. So I
>>>>>>>>> think
>>>>>>>>> there were
>>>>>>>>> no revolutionary changes anywhere. My hope was (and still
>>>>>>>>> is),
>>>>>>>>> that anybody
>>>>>>>>> here on the list might have had a similar problem and has
>>>>>>>>> solved
>>>>>>>>> it, or
>>>>>>>>> maybe someone knows about some changes in Dev mode which
>>>>>>>>> could
>>>>>>>>> explain such
>>>>>>>>> behaviour (don't know, maybe xml parsing or component tree
>>>>>>>>> generation has
>>>>>>>>> changed, or scope handling of managed beans, or whatever
>>>>>>>>> could
>>>>>>>>> cause some
>>>>>>>>> code to be entered far more often than before, or some
>>>>>>>>> cache/pool
>>>>>>>>> was
>>>>>>>>> removed or has different behaviour... such things). Then it
>>>>>>>>> would
>>>>>>>>> be far
>>>>>>>>> more easy for me to track down the real problem.
>>>>>>>>>>> Anyway, when I get some spare time (currently have some
>>>>>>>>>>> deadlines in
>>>>>>>>> sight), I'll try to build a small test case. Either I can get
>>>>>>>>> a
>>>>>>>>> grip on the
>>>>>>>>> issue myself on doing this, or I can supply it to anyone who
>>>>>>>>> might want to
>>>>>>>>> have a look at it.
>>>>>>>>>>> cu
>>>>>>>>>>> Jens
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
>>>>>>>>>>>>> newer versions are so incredibly slow in dev mode
>>>>>>>>>>>> I can assure you that Dev/Prod mode works very very
>>>>>>>>>>>> swiftly
>>>>>>>>>>>> on 8.0.15
>>>>>>>>>>>> for sizable applications. For example, I have a giant
>>>>>>>>>>>> application
>>>>>>>>>>>> (thousands of LOC, 50+ jsf views) that has sub-8ms
>>>>>>>>>>>> response
>>>>>>>>>>>> times
>>>>>>>>>>>> (minus database time). More than likely they are doing
>>>>>>>>>>>> something they
>>>>>>>>>>>> should not be doing with JSF apis and are causing
>>>>>>>>>>>> problems.
>>>>>>>>>>>>
>>>>>>>>>>>> I would check the obvious things first with VisualVM:
>>>>>>>>>>>> heap
>>>>>>>>>>>> size, gc
>>>>>>>>>>>> frequency, etc. You can also run their profiler and see
>>>>>>>>>>>> if
>>>>>>>>>>>> you can
>>>>>>>>>>>> spot anything immediately obvious.
>>>>>>>>>>>>
>>>>>>>>>>>> After that, the easiest way to locate the code slowing
>>>>>>>>>>>> you
>>>>>>>>>>>> down is
>>>>>>>>>>>> bisection. Cut half the code out, run, and continue
>>>>>>>>>>>> cutting
>>>>>>>>>>>> until you
>>>>>>>>>>>> can locate the problematic code. Then after that,
>>>>>>>>>>>> isolate a
>>>>>>>>>>>> test case
>>>>>>>>>>>> where flipping a boolean flag triggers the issue and
>>>>>>>>>>>> post
>>>>>>>>>>>> the results
>>>>>>>>>>>> here.
>>>>>>>>>>>>
>>>>>>>>>>>> I would get ahold of: https://www.tomitribe.com who are
>>>>>>>>>>>> literal
>>>>>>>>>>>> experts in this stuff. They offer professional services
>>>>>>>>>>>> to
>>>>>>>>>>>> handle
>>>>>>>>>>>> exactly these situations and can work directly in your
>>>>>>>>>>>> codebase to
>>>>>>>>>>>> help you find and fix the problem.
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski
>>>>>>>>>>>> <jz...@diegurus.de> wrote:
>>>>>>>>>>>>> Hi altogether,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm new to this list and hope it's the right place to
>>>>>>>>>>>>> ask
>>>>>>>>>>>>> this
>>>>>>>>> question.
>>>>>>>>>>>>> If not, and someone know the right place to ask this,
>>>>>>>>>>>>> please give
>>>>>>>>> advise.
>>>>>>>>>>>>> I'm developing a big JSF Application for a customer.
>>>>>>>>>>>>> It's
>>>>>>>>>>>>> a long
>>>>>>>>> running
>>>>>>>>>>>>> project and development and when I started with it,
>>>>>>>>>>>>> TomEE
>>>>>>>>>>>>> 8.0.6 was
>>>>>>>>> the
>>>>>>>>>>>>> most recent Version of TomEE. The switch to Jakarta
>>>>>>>>>>>>> is
>>>>>>>>>>>>> planned for
>>>>>>>>> next
>>>>>>>>>>>>> year, so I'm still on the 8.x path for the time being
>>>>>>>>>>>>> and
>>>>>>>>>>>>> wasn't able
>>>>>>>>> to
>>>>>>>>>>>>> test if this problem still exists in the 9.x branch.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The problem:
>>>>>>>>>>>>> On the customers site I'm with the latest TomEE
>>>>>>>>>>>>> version
>>>>>>>>>>>>> 8.0.15
>>>>>>>>> (running
>>>>>>>>>>>>> on Java 11) and everything works fine, because it is
>>>>>>>>>>>>> running in
>>>>>>>>>>>>> production mode. But in my development environment
>>>>>>>>>>>>> I'm
>>>>>>>>>>>>> still stuck
>>>>>>>>> with
>>>>>>>>>>>>> 8.0.6, because there I need the development mode
>>>>>>>>>>>>> (javax.faces.PROJECT_STAGE: Development). I need to
>>>>>>>>>>>>> be
>>>>>>>>>>>>> able to see
>>>>>>>>>>>>> changes in facelets without restarting everything
>>>>>>>>>>>>> everytime, and to
>>>>>>>>> get
>>>>>>>>>>>>> more detailed error messages.
>>>>>>>>>>>>> My attempts to update my dev TomEE to something newer
>>>>>>>>>>>>> than 8.0.6 all
>>>>>>>>>>>>> failed, because all (at least all I've tested so far)
>>>>>>>>>>>>> newer versions
>>>>>>>>> are
>>>>>>>>>>>>> so incredibly slow in dev mode, that it's unbearable
>>>>>>>>>>>>> to
>>>>>>>>>>>>> use the
>>>>>>>>>>>>> application. Several very looong seconds on every
>>>>>>>>>>>>> request
>>>>>>>>>>>>> (even the
>>>>>>>>>>>>> little AJAX requests in a view) is simply not a
>>>>>>>>>>>>> practical
>>>>>>>>>>>>> environment.
>>>>>>>>>>>>> After switching to Production mode everything works
>>>>>>>>>>>>> very
>>>>>>>>>>>>> fast, and all
>>>>>>>>>>>>> my views have response times of very few ms (even the
>>>>>>>>>>>>> big
>>>>>>>>>>>>> ones with
>>>>>>>>> max.
>>>>>>>>>>>>> around 400ms). When switching back to Development
>>>>>>>>>>>>> mode I
>>>>>>>>>>>>> have response
>>>>>>>>>>>>> times of up to 30s on big views. Not with 8.0.6,
>>>>>>>>>>>>> there
>>>>>>>>>>>>> even in
>>>>>>>>>>>>> development mode it's reasonable fast in not getting
>>>>>>>>>>>>> higher than 1s.
>>>>>>>>>>>>>
>>>>>>>>>>>>> My question:
>>>>>>>>>>>>> What causes this enormous performance degradation?
>>>>>>>>>>>>> I'm
>>>>>>>>>>>>> hoping, it is
>>>>>>>>>>>>> just a configuration which now has another default
>>>>>>>>>>>>> value
>>>>>>>>>>>>> or the like.
>>>>>>>>> If
>>>>>>>>>>>>> yes, maybe someone can point me in the right
>>>>>>>>>>>>> direction of
>>>>>>>>>>>>> where to
>>>>>>>>> find
>>>>>>>>>>>>> this configuration? If it's not a configuration
>>>>>>>>>>>>> thing:
>>>>>>>>>>>>> what can I do
>>>>>>>>> to
>>>>>>>>>>>>> get around this?
>>>>>>>>>>>>>
>>>>>>>>>>>>> I haven't tested all Versions of TomEE from 8.0.6 to
>>>>>>>>>>>>> 8.0.15, so I
>>>>>>>>> can't
>>>>>>>>>>>>> say at what version exactly this behaviour changes.
>>>>>>>>>>>>> If it
>>>>>>>>>>>>> helps or is
>>>>>>>>>>>>> needed, I can make some tests to find the exact
>>>>>>>>>>>>> version
>>>>>>>>>>>>> where this
>>>>>>>>>>>>> happens. Versions I've already tested are: 8.0.12,
>>>>>>>>>>>>> 8.0.13
>>>>>>>>>>>>> and 8.0.15.
>>>>>>>>>>>>> All of them are very slow in dev mode. If you need
>>>>>>>>>>>>> more
>>>>>>>>>>>>> information,
>>>>>>>>>>>>> I'll try to provide it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks in advance for any help
>>>>>>>>>>>>> cu
>>>>>>>>>>>>> Jens
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Vicente Rossello <co...@gmail.com>.
I want through this, now I remember. Try with these system switches
-Dsun.io.useCanonCaches=true -Dsun.io.useCanonPrefixCache=true
Dont use this in production, be aware.
We ended up removing these flash and using a dirty trick with tomcat
DirResourcesSet
I can send you this tonight. Try the flags to see if we are talking about
the same problem
El vie, 27 oct 2023, 16:56, Jens Zurawski <jz...@diegurus.de> escribió:
> I have a first result on this issue:
>
> The problem does not occur if TomEE is running on Linux. I've tested my
> own built 8.0.8, a freshly extracted 8.0.8 and 8.0.15 from the releases,
> and none of them showed the dev mode performance problem.
>
> Using TomEE 8.0.8 on Windows does show the dev mode performance problem.
> All of the tested ones, means the released 8.0.8 and my own built 8.0.8
> (the one I've built on Linux).
>
> My development environment is Windows 10 (Netbeans and therefore also
> the TomEE for easy deployment and debugging). The problem does occur on
> stand alone started TomEE and also if started out of Netbeans, so
> Netbeans can be sorted out as the root of the problem. Java Versions 11,
> 13, 15 and 17 tested, all the same, so Java should also not be the
> problem. BTW: It is very unlikely a problem of my individual Windows
> installation, because this problem also shows up on the Windows system
> of my colleague, which btw. was the root cause why I'm started to dig
> deeper into this problem now and why I subscribed to this mailing list ;-)
>
> I'm afraid, this will complicate my git bisect session a little bit :-(
>
> However, I'll keep you posted.
>
> cu
> Jens
>
>
> Am 27.10.2023 um 13:12 schrieb Jens Zurawski:
> > Thanks Richard (and Vicente),
> >
> > I've changed to JDK 8 and now the builds are successful (both 8.0.6
> > and 8.0.8).
> >
> > I'll script me some test scripts to ease the build, run and test for
> > every bisect step.
> > A build needs around 3 to 4 minutes. Running a test with my
> > application will need some manual interaction and checks, I guess also
> > something around 4 minutes. This sums up to roughly 8 minutes per step.
> >
> > I've started a bisect and it told me:
> > git bisect good 20441eb
> > git bisect bad 4c8a616
> > Bisecting: 136 revisions left to test after this (roughly 7 steps)
> >
> > That means, if everything works fine, I should be able to identify the
> > commit after about an hour? (That's cool.... why nobody has told me
> > about git bisect before? ;-) )
> >
> > Anyway, I will have to find some time for this, maybe I need a couple
> > of days to find some.
> > I will let you know about the outcome.
> >
> > cu
> > Jens
> >
> >
> > Am 27.10.2023 um 11:50 schrieb Richard Zowalla:
> >> Hi,
> >>
> >> you need Java/JDK 8 to build TomEE 8.
> >>
> >> The Maven command is
> >>
> >> mvn --fail-at-end clean install -pl tomee/apache-tomee -am -
> >> Dfile.encoding=UTF-8 -DskipTests=true
> >>
> >> You will find the tar.gz/zip in ./tomee/apache-tomee/target
> >>
> >> Gruß
> >> Richard
> >>
> >> P.S. 8.0.7 is here: https://archive.apache.org/dist/tomee/tomee-8.0.7/
> >>
> >> Am Freitag, dem 27.10.2023 um 10:50 +0200 schrieb Jens Zurawski:
> >>> Hi Richard,
> >>>
> >>> to identify the "bad" version was easy. I didn't find a 8.0.7 release
> >>> on
> >>> the download page, so I started with 8.0.8. And this version already
> >>> has
> >>> the slow dev mode. So for my problem I can say:
> >>> good: 8.0.6
> >>> bad: 8.0.8
> >>>
> >>> Now, I think, I need a little help from you. I tried to build a TomEE
> >>> out of git (I've cloned the github repository
> >>> https://github.com/apache/tomee.git). I've tried these two commits:
> >>> 20441eb (tag: tomee-8.0.6)
> >>> 4c8a616 (tag: tomee-project-8.0.8)
> >>>
> >>> But the build fails every time on "TomEE :: Container :: Core" with:
> >>>
> >>> [INFO] BUILD FAILURE
> >>> [INFO]
> >>> ---------------------------------------------------------------------
> >>> ---
> >>> [INFO] Total time: 01:19 min
> >>> [INFO] Finished at: 2023-10-27T10:29:40+02:00
> >>> [INFO]
> >>> ---------------------------------------------------------------------
> >>> ---
> >>> [ERROR] Failed to execute goal
> >>> org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile
> >>> (default-compile) on project openejb-core: Compilation failure:
> >>> Compilation failure:
> >>> [ERROR]
> >>> /sandbox/tomee/container/openejb-
> >>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[21,21]
> >>> package org.omg.CORBA does not exist
> >>> [ERROR]
> >>> /sandbox/tomee/container/openejb-
> >>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[24,12]
> >>> cannot find symbol
> >>> [ERROR] symbol: class ORB
> >>> [ERROR] location: class org.apache.openejb.core.OrbFactory
> >>>
> >>> I've tried to build with:
> >>> mvn clean install
> >>>
> >>> My build environment is a linux ubuntu 20.04.6 LTS VM with:
> >>> git 2.25.1
> >>> Maven 3.6.3
> >>> java 14.0.2
> >>>
> >>> Is it some easy thing to fix, or do I have to switch some versions of
> >>> my
> >>> environment?
> >>>
> >>> cu
> >>> Jens
> >>>
> >>> Am 26.10.2023 um 19:48 schrieb Richard Zowalla:
> >>>> Hi,
> >>>>
> >>>> without knowing code or having a reproducer, it might be not that
> >>>> obvious to find a reason / answer for the "academic problem" ;-)
> >>>>
> >>>> If you really want to boil it down, I recommend the following
> >>>> approach:
> >>>>
> >>>> (1) From your description it seems, that 8.0.6 is the last known
> >>>> "good"
> >>>> version.
> >>>>
> >>>> (2) You've investigated versions starting from 8.0.12 onwards,
> >>>> which
> >>>> seems to be "bad".
> >>>>
> >>>> Following the approach recommended by Jonathan:
> >>>>
> >>>> - Can you check if it happens in 8.0.7 (8.0.8, ...) until we find
> >>>> the
> >>>> first "bad" released version?
> >>>>
> >>>> - If we have that, you could start a "git bisect" session between
> >>>> 8.0.6
> >>>> and the first version, which is known to be bad (reducing the
> >>>> amount of
> >>>> needed bisect cycles dramatically), so we find a commit / reason
> >>>> (if it
> >>>> is in TomEE).
> >>>>
> >>>> - It would involve some quick builds without executing tests (~10-
> >>>> 15min
> >>>> each) for each bisect step and testing on your side.
> >>>>
> >>>> Don't know if your inner monk is willing to go that route, but I am
> >>>> happy to help with the related Maven commands for quick building :)
> >>>>
> >>>> Gruß
> >>>> Richard
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> Am Donnerstag, dem 26.10.2023 um 18:00 +0200 schrieb Jens Zurawski:
> >>>>> Hi Vicente,
> >>>>>
> >>>>> thank you for the tip. This is a good workaround. When I switch
> >>>>> to
> >>>>> Production mode and insert this configuration, it reloads the
> >>>>> view
> >>>>> from
> >>>>> the file if it has changed. This means, I can use a decent
> >>>>> version of
> >>>>> TomEE in my dev environment.
> >>>>>
> >>>>> However, it doesn't completely solve my problem, as the error
> >>>>> reporting
> >>>>> isn't as verbose as in Dev mode as far as I can see. But I have
> >>>>> to
> >>>>> check, maybe it's enough for my development.
> >>>>>
> >>>>> And, well.... the academic problem: "Why is it still broken in
> >>>>> Dev
> >>>>> mode?" still runs circles through my brain ;-) Because if it's a
> >>>>> problem
> >>>>> of my coding, it might be a good idea to change this, before it
> >>>>> might
> >>>>> get a problem on Production mode, too, at some time in the
> >>>>> future.
> >>>>>
> >>>>> But anyway, you've helped me a good step forward, thanks.
> >>>>> cu
> >>>>> Jens
> >>>>>
> >>>>>
> >>>>> Am 26.10.2023 um 17:04 schrieb Vicente Rossello:
> >>>>>> We are using the production stage with
> >>>>>> javax.faces.FACELETS_REFRESH_PERIOD
> >>>>>> = 1. It's faster and more similar to production, just try it if
> >>>>>> it
> >>>>>> solves
> >>>>>> your problem
> >>>>>>
> >>>>>> On Thu, Oct 26, 2023 at 4:18 PM Jens Zurawski <jz...@diegurus.de>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> The one bundled with TomEE.
> >>>>>>> myfaces 2.3.7 on TomEE 8.0.6
> >>>>>>> and e.g. myfaces 2.3.9 on TomEE 8.0.12
> >>>>>>>
> >>>>>>> Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
> >>>>>>>> Which JSF lib are you using?
> >>>>>>>>
> >>>>>>>> Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens Zurawski
> >>>>>>>> <jzu@diegurus.de
> >>>>>>>> :
> >>>>>>>>> Thank you for your quick reply, Jonathan.
> >>>>>>>>>
> >>>>>>>>> It is good to know, that normally the Dev mode should
> >>>>>>>>> work as
> >>>>>>>>> fast as
> >>>>>>> before even with 8.0.15. So it obviously has something to do
> >>>>>>> with
> >>>>>>> my
> >>>>>>> environment, configuration and/or code.
> >>>>>>>>> And also thank you for the reference to tomitribe.
> >>>>>>>>> Unfortunately I'm
> >>>>>>> the only developer on this project, and it will be difficult
> >>>>>>> to
> >>>>>>> convince my
> >>>>>>> customer to an additional support subscription, because he on
> >>>>>>> production
> >>>>>>> doesn't have issues ;-) But, ok, that's my problem, and maybe
> >>>>>>> I'll consult
> >>>>>>> tomitribe in the future for other projects.
> >>>>>>>>> So, I'm afraid, for the near future I'm on my own to
> >>>>>>>>> solve
> >>>>>>>>> this
> >>>>>>> problem. Therefore any hints to restrict the places where I
> >>>>>>> should take a
> >>>>>>> look will be more than welcome.
> >>>>>>>>> I'm a senior programmer with 20 years of experience in
> >>>>>>>>> Java
> >>>>>>>>> programming
> >>>>>>> (SE and JSP). But I'm relative new to JSF/JavaEE/JakartaEE
> >>>>>>> and
> >>>>>>> sometimes I
> >>>>>>> have a hard time to understand which component of the whole
> >>>>>>> stack
> >>>>>>> is
> >>>>>>> responsible for what.
> >>>>>>>>> The project meanwhile has over a 1/4 million lines of
> >>>>>>>>> code
> >>>>>>>>> (most of
> >>>>>>> them in the back-end, but also the front-end is big enough to
> >>>>>>> not
> >>>>>>> trace
> >>>>>>> through every lane if I want to get to the point in this life
> >>>>>>> :-D
> >>>>>>> ).
> >>>>>>>>> The application now runs for nearly 2 years on the
> >>>>>>>>> customers
> >>>>>>>>> site and
> >>>>>>> is very reliable and fast in its daily usage. Apart from
> >>>>>>> occasional updates
> >>>>>>> every few month, it will never be restarted or need any
> >>>>>>> manual
> >>>>>>> maintenance
> >>>>>>> at all. So I assume the code is not the badest on earth. At
> >>>>>>> least
> >>>>>>> because
> >>>>>>> of that I'm able to out-rule things like memory leaks, gc
> >>>>>>> issues
> >>>>>>> or
> >>>>>>> bottlenecks (max Heap is never reached, neither half of it,
> >>>>>>> even
> >>>>>>> after
> >>>>>>> running for month).
> >>>>>>>>> Of course there could be aspects of JSF which I may not
> >>>>>>>>> have
> >>>>>>>>> understood
> >>>>>>> correctly until now. But because there are so many new things
> >>>>>>> to
> >>>>>>> a JSF
> >>>>>>> beginner like me, I'm running out of ideas what this could
> >>>>>>> be. It
> >>>>>>> is
> >>>>>>> something that worked well 'til TomEE 8.0.6 (Prod and Dev
> >>>>>>> mode)
> >>>>>>> and
> >>>>>>> suddenly don't work anymore on 8.0.12 in Dev mode (but still
> >>>>>>> works great in
> >>>>>>> Prod mode). There are only minor versions in-between. So I
> >>>>>>> think
> >>>>>>> there were
> >>>>>>> no revolutionary changes anywhere. My hope was (and still
> >>>>>>> is),
> >>>>>>> that anybody
> >>>>>>> here on the list might have had a similar problem and has
> >>>>>>> solved
> >>>>>>> it, or
> >>>>>>> maybe someone knows about some changes in Dev mode which
> >>>>>>> could
> >>>>>>> explain such
> >>>>>>> behaviour (don't know, maybe xml parsing or component tree
> >>>>>>> generation has
> >>>>>>> changed, or scope handling of managed beans, or whatever
> >>>>>>> could
> >>>>>>> cause some
> >>>>>>> code to be entered far more often than before, or some
> >>>>>>> cache/pool
> >>>>>>> was
> >>>>>>> removed or has different behaviour... such things). Then it
> >>>>>>> would
> >>>>>>> be far
> >>>>>>> more easy for me to track down the real problem.
> >>>>>>>>> Anyway, when I get some spare time (currently have some
> >>>>>>>>> deadlines in
> >>>>>>> sight), I'll try to build a small test case. Either I can get
> >>>>>>> a
> >>>>>>> grip on the
> >>>>>>> issue myself on doing this, or I can supply it to anyone who
> >>>>>>> might want to
> >>>>>>> have a look at it.
> >>>>>>>>> cu
> >>>>>>>>> Jens
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
> >>>>>>>>>>> newer versions are so incredibly slow in dev mode
> >>>>>>>>>> I can assure you that Dev/Prod mode works very very
> >>>>>>>>>> swiftly
> >>>>>>>>>> on 8.0.15
> >>>>>>>>>> for sizable applications. For example, I have a giant
> >>>>>>>>>> application
> >>>>>>>>>> (thousands of LOC, 50+ jsf views) that has sub-8ms
> >>>>>>>>>> response
> >>>>>>>>>> times
> >>>>>>>>>> (minus database time). More than likely they are doing
> >>>>>>>>>> something they
> >>>>>>>>>> should not be doing with JSF apis and are causing
> >>>>>>>>>> problems.
> >>>>>>>>>>
> >>>>>>>>>> I would check the obvious things first with VisualVM:
> >>>>>>>>>> heap
> >>>>>>>>>> size, gc
> >>>>>>>>>> frequency, etc. You can also run their profiler and see
> >>>>>>>>>> if
> >>>>>>>>>> you can
> >>>>>>>>>> spot anything immediately obvious.
> >>>>>>>>>>
> >>>>>>>>>> After that, the easiest way to locate the code slowing
> >>>>>>>>>> you
> >>>>>>>>>> down is
> >>>>>>>>>> bisection. Cut half the code out, run, and continue
> >>>>>>>>>> cutting
> >>>>>>>>>> until you
> >>>>>>>>>> can locate the problematic code. Then after that,
> >>>>>>>>>> isolate a
> >>>>>>>>>> test case
> >>>>>>>>>> where flipping a boolean flag triggers the issue and
> >>>>>>>>>> post
> >>>>>>>>>> the results
> >>>>>>>>>> here.
> >>>>>>>>>>
> >>>>>>>>>> I would get ahold of: https://www.tomitribe.com who are
> >>>>>>>>>> literal
> >>>>>>>>>> experts in this stuff. They offer professional services
> >>>>>>>>>> to
> >>>>>>>>>> handle
> >>>>>>>>>> exactly these situations and can work directly in your
> >>>>>>>>>> codebase to
> >>>>>>>>>> help you find and fix the problem.
> >>>>>>>>>>
> >>>>>>>>>> On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski
> >>>>>>>>>> <jz...@diegurus.de> wrote:
> >>>>>>>>>>> Hi altogether,
> >>>>>>>>>>>
> >>>>>>>>>>> I'm new to this list and hope it's the right place to
> >>>>>>>>>>> ask
> >>>>>>>>>>> this
> >>>>>>> question.
> >>>>>>>>>>> If not, and someone know the right place to ask this,
> >>>>>>>>>>> please give
> >>>>>>> advise.
> >>>>>>>>>>> I'm developing a big JSF Application for a customer.
> >>>>>>>>>>> It's
> >>>>>>>>>>> a long
> >>>>>>> running
> >>>>>>>>>>> project and development and when I started with it,
> >>>>>>>>>>> TomEE
> >>>>>>>>>>> 8.0.6 was
> >>>>>>> the
> >>>>>>>>>>> most recent Version of TomEE. The switch to Jakarta
> >>>>>>>>>>> is
> >>>>>>>>>>> planned for
> >>>>>>> next
> >>>>>>>>>>> year, so I'm still on the 8.x path for the time being
> >>>>>>>>>>> and
> >>>>>>>>>>> wasn't able
> >>>>>>> to
> >>>>>>>>>>> test if this problem still exists in the 9.x branch.
> >>>>>>>>>>>
> >>>>>>>>>>> The problem:
> >>>>>>>>>>> On the customers site I'm with the latest TomEE
> >>>>>>>>>>> version
> >>>>>>>>>>> 8.0.15
> >>>>>>> (running
> >>>>>>>>>>> on Java 11) and everything works fine, because it is
> >>>>>>>>>>> running in
> >>>>>>>>>>> production mode. But in my development environment
> >>>>>>>>>>> I'm
> >>>>>>>>>>> still stuck
> >>>>>>> with
> >>>>>>>>>>> 8.0.6, because there I need the development mode
> >>>>>>>>>>> (javax.faces.PROJECT_STAGE: Development). I need to
> >>>>>>>>>>> be
> >>>>>>>>>>> able to see
> >>>>>>>>>>> changes in facelets without restarting everything
> >>>>>>>>>>> everytime, and to
> >>>>>>> get
> >>>>>>>>>>> more detailed error messages.
> >>>>>>>>>>> My attempts to update my dev TomEE to something newer
> >>>>>>>>>>> than 8.0.6 all
> >>>>>>>>>>> failed, because all (at least all I've tested so far)
> >>>>>>>>>>> newer versions
> >>>>>>> are
> >>>>>>>>>>> so incredibly slow in dev mode, that it's unbearable
> >>>>>>>>>>> to
> >>>>>>>>>>> use the
> >>>>>>>>>>> application. Several very looong seconds on every
> >>>>>>>>>>> request
> >>>>>>>>>>> (even the
> >>>>>>>>>>> little AJAX requests in a view) is simply not a
> >>>>>>>>>>> practical
> >>>>>>>>>>> environment.
> >>>>>>>>>>> After switching to Production mode everything works
> >>>>>>>>>>> very
> >>>>>>>>>>> fast, and all
> >>>>>>>>>>> my views have response times of very few ms (even the
> >>>>>>>>>>> big
> >>>>>>>>>>> ones with
> >>>>>>> max.
> >>>>>>>>>>> around 400ms). When switching back to Development
> >>>>>>>>>>> mode I
> >>>>>>>>>>> have response
> >>>>>>>>>>> times of up to 30s on big views. Not with 8.0.6,
> >>>>>>>>>>> there
> >>>>>>>>>>> even in
> >>>>>>>>>>> development mode it's reasonable fast in not getting
> >>>>>>>>>>> higher than 1s.
> >>>>>>>>>>>
> >>>>>>>>>>> My question:
> >>>>>>>>>>> What causes this enormous performance degradation?
> >>>>>>>>>>> I'm
> >>>>>>>>>>> hoping, it is
> >>>>>>>>>>> just a configuration which now has another default
> >>>>>>>>>>> value
> >>>>>>>>>>> or the like.
> >>>>>>> If
> >>>>>>>>>>> yes, maybe someone can point me in the right
> >>>>>>>>>>> direction of
> >>>>>>>>>>> where to
> >>>>>>> find
> >>>>>>>>>>> this configuration? If it's not a configuration
> >>>>>>>>>>> thing:
> >>>>>>>>>>> what can I do
> >>>>>>> to
> >>>>>>>>>>> get around this?
> >>>>>>>>>>>
> >>>>>>>>>>> I haven't tested all Versions of TomEE from 8.0.6 to
> >>>>>>>>>>> 8.0.15, so I
> >>>>>>> can't
> >>>>>>>>>>> say at what version exactly this behaviour changes.
> >>>>>>>>>>> If it
> >>>>>>>>>>> helps or is
> >>>>>>>>>>> needed, I can make some tests to find the exact
> >>>>>>>>>>> version
> >>>>>>>>>>> where this
> >>>>>>>>>>> happens. Versions I've already tested are: 8.0.12,
> >>>>>>>>>>> 8.0.13
> >>>>>>>>>>> and 8.0.15.
> >>>>>>>>>>> All of them are very slow in dev mode. If you need
> >>>>>>>>>>> more
> >>>>>>>>>>> information,
> >>>>>>>>>>> I'll try to provide it.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks in advance for any help
> >>>>>>>>>>> cu
> >>>>>>>>>>> Jens
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>
>
>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Jens Zurawski <jz...@diegurus.de>.
I have a first result on this issue:
The problem does not occur if TomEE is running on Linux. I've tested my
own built 8.0.8, a freshly extracted 8.0.8 and 8.0.15 from the releases,
and none of them showed the dev mode performance problem.
Using TomEE 8.0.8 on Windows does show the dev mode performance problem.
All of the tested ones, means the released 8.0.8 and my own built 8.0.8
(the one I've built on Linux).
My development environment is Windows 10 (Netbeans and therefore also
the TomEE for easy deployment and debugging). The problem does occur on
stand alone started TomEE and also if started out of Netbeans, so
Netbeans can be sorted out as the root of the problem. Java Versions 11,
13, 15 and 17 tested, all the same, so Java should also not be the
problem. BTW: It is very unlikely a problem of my individual Windows
installation, because this problem also shows up on the Windows system
of my colleague, which btw. was the root cause why I'm started to dig
deeper into this problem now and why I subscribed to this mailing list ;-)
I'm afraid, this will complicate my git bisect session a little bit :-(
However, I'll keep you posted.
cu
Jens
Am 27.10.2023 um 13:12 schrieb Jens Zurawski:
> Thanks Richard (and Vicente),
>
> I've changed to JDK 8 and now the builds are successful (both 8.0.6
> and 8.0.8).
>
> I'll script me some test scripts to ease the build, run and test for
> every bisect step.
> A build needs around 3 to 4 minutes. Running a test with my
> application will need some manual interaction and checks, I guess also
> something around 4 minutes. This sums up to roughly 8 minutes per step.
>
> I've started a bisect and it told me:
> git bisect good 20441eb
> git bisect bad 4c8a616
> Bisecting: 136 revisions left to test after this (roughly 7 steps)
>
> That means, if everything works fine, I should be able to identify the
> commit after about an hour? (That's cool.... why nobody has told me
> about git bisect before? ;-) )
>
> Anyway, I will have to find some time for this, maybe I need a couple
> of days to find some.
> I will let you know about the outcome.
>
> cu
> Jens
>
>
> Am 27.10.2023 um 11:50 schrieb Richard Zowalla:
>> Hi,
>>
>> you need Java/JDK 8 to build TomEE 8.
>>
>> The Maven command is
>>
>> mvn --fail-at-end clean install -pl tomee/apache-tomee -am -
>> Dfile.encoding=UTF-8 -DskipTests=true
>>
>> You will find the tar.gz/zip in ./tomee/apache-tomee/target
>>
>> Gruß
>> Richard
>>
>> P.S. 8.0.7 is here: https://archive.apache.org/dist/tomee/tomee-8.0.7/
>>
>> Am Freitag, dem 27.10.2023 um 10:50 +0200 schrieb Jens Zurawski:
>>> Hi Richard,
>>>
>>> to identify the "bad" version was easy. I didn't find a 8.0.7 release
>>> on
>>> the download page, so I started with 8.0.8. And this version already
>>> has
>>> the slow dev mode. So for my problem I can say:
>>> good: 8.0.6
>>> bad: 8.0.8
>>>
>>> Now, I think, I need a little help from you. I tried to build a TomEE
>>> out of git (I've cloned the github repository
>>> https://github.com/apache/tomee.git). I've tried these two commits:
>>> 20441eb (tag: tomee-8.0.6)
>>> 4c8a616 (tag: tomee-project-8.0.8)
>>>
>>> But the build fails every time on "TomEE :: Container :: Core" with:
>>>
>>> [INFO] BUILD FAILURE
>>> [INFO]
>>> ---------------------------------------------------------------------
>>> ---
>>> [INFO] Total time: 01:19 min
>>> [INFO] Finished at: 2023-10-27T10:29:40+02:00
>>> [INFO]
>>> ---------------------------------------------------------------------
>>> ---
>>> [ERROR] Failed to execute goal
>>> org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile
>>> (default-compile) on project openejb-core: Compilation failure:
>>> Compilation failure:
>>> [ERROR]
>>> /sandbox/tomee/container/openejb-
>>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[21,21]
>>> package org.omg.CORBA does not exist
>>> [ERROR]
>>> /sandbox/tomee/container/openejb-
>>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[24,12]
>>> cannot find symbol
>>> [ERROR] symbol: class ORB
>>> [ERROR] location: class org.apache.openejb.core.OrbFactory
>>>
>>> I've tried to build with:
>>> mvn clean install
>>>
>>> My build environment is a linux ubuntu 20.04.6 LTS VM with:
>>> git 2.25.1
>>> Maven 3.6.3
>>> java 14.0.2
>>>
>>> Is it some easy thing to fix, or do I have to switch some versions of
>>> my
>>> environment?
>>>
>>> cu
>>> Jens
>>>
>>> Am 26.10.2023 um 19:48 schrieb Richard Zowalla:
>>>> Hi,
>>>>
>>>> without knowing code or having a reproducer, it might be not that
>>>> obvious to find a reason / answer for the "academic problem" ;-)
>>>>
>>>> If you really want to boil it down, I recommend the following
>>>> approach:
>>>>
>>>> (1) From your description it seems, that 8.0.6 is the last known
>>>> "good"
>>>> version.
>>>>
>>>> (2) You've investigated versions starting from 8.0.12 onwards,
>>>> which
>>>> seems to be "bad".
>>>>
>>>> Following the approach recommended by Jonathan:
>>>>
>>>> - Can you check if it happens in 8.0.7 (8.0.8, ...) until we find
>>>> the
>>>> first "bad" released version?
>>>>
>>>> - If we have that, you could start a "git bisect" session between
>>>> 8.0.6
>>>> and the first version, which is known to be bad (reducing the
>>>> amount of
>>>> needed bisect cycles dramatically), so we find a commit / reason
>>>> (if it
>>>> is in TomEE).
>>>>
>>>> - It would involve some quick builds without executing tests (~10-
>>>> 15min
>>>> each) for each bisect step and testing on your side.
>>>>
>>>> Don't know if your inner monk is willing to go that route, but I am
>>>> happy to help with the related Maven commands for quick building :)
>>>>
>>>> Gruß
>>>> Richard
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Am Donnerstag, dem 26.10.2023 um 18:00 +0200 schrieb Jens Zurawski:
>>>>> Hi Vicente,
>>>>>
>>>>> thank you for the tip. This is a good workaround. When I switch
>>>>> to
>>>>> Production mode and insert this configuration, it reloads the
>>>>> view
>>>>> from
>>>>> the file if it has changed. This means, I can use a decent
>>>>> version of
>>>>> TomEE in my dev environment.
>>>>>
>>>>> However, it doesn't completely solve my problem, as the error
>>>>> reporting
>>>>> isn't as verbose as in Dev mode as far as I can see. But I have
>>>>> to
>>>>> check, maybe it's enough for my development.
>>>>>
>>>>> And, well.... the academic problem: "Why is it still broken in
>>>>> Dev
>>>>> mode?" still runs circles through my brain ;-) Because if it's a
>>>>> problem
>>>>> of my coding, it might be a good idea to change this, before it
>>>>> might
>>>>> get a problem on Production mode, too, at some time in the
>>>>> future.
>>>>>
>>>>> But anyway, you've helped me a good step forward, thanks.
>>>>> cu
>>>>> Jens
>>>>>
>>>>>
>>>>> Am 26.10.2023 um 17:04 schrieb Vicente Rossello:
>>>>>> We are using the production stage with
>>>>>> javax.faces.FACELETS_REFRESH_PERIOD
>>>>>> = 1. It's faster and more similar to production, just try it if
>>>>>> it
>>>>>> solves
>>>>>> your problem
>>>>>>
>>>>>> On Thu, Oct 26, 2023 at 4:18 PM Jens Zurawski <jz...@diegurus.de>
>>>>>> wrote:
>>>>>>
>>>>>>> The one bundled with TomEE.
>>>>>>> myfaces 2.3.7 on TomEE 8.0.6
>>>>>>> and e.g. myfaces 2.3.9 on TomEE 8.0.12
>>>>>>>
>>>>>>> Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
>>>>>>>> Which JSF lib are you using?
>>>>>>>>
>>>>>>>> Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens Zurawski
>>>>>>>> <jzu@diegurus.de
>>>>>>>> :
>>>>>>>>> Thank you for your quick reply, Jonathan.
>>>>>>>>>
>>>>>>>>> It is good to know, that normally the Dev mode should
>>>>>>>>> work as
>>>>>>>>> fast as
>>>>>>> before even with 8.0.15. So it obviously has something to do
>>>>>>> with
>>>>>>> my
>>>>>>> environment, configuration and/or code.
>>>>>>>>> And also thank you for the reference to tomitribe.
>>>>>>>>> Unfortunately I'm
>>>>>>> the only developer on this project, and it will be difficult
>>>>>>> to
>>>>>>> convince my
>>>>>>> customer to an additional support subscription, because he on
>>>>>>> production
>>>>>>> doesn't have issues ;-) But, ok, that's my problem, and maybe
>>>>>>> I'll consult
>>>>>>> tomitribe in the future for other projects.
>>>>>>>>> So, I'm afraid, for the near future I'm on my own to
>>>>>>>>> solve
>>>>>>>>> this
>>>>>>> problem. Therefore any hints to restrict the places where I
>>>>>>> should take a
>>>>>>> look will be more than welcome.
>>>>>>>>> I'm a senior programmer with 20 years of experience in
>>>>>>>>> Java
>>>>>>>>> programming
>>>>>>> (SE and JSP). But I'm relative new to JSF/JavaEE/JakartaEE
>>>>>>> and
>>>>>>> sometimes I
>>>>>>> have a hard time to understand which component of the whole
>>>>>>> stack
>>>>>>> is
>>>>>>> responsible for what.
>>>>>>>>> The project meanwhile has over a 1/4 million lines of
>>>>>>>>> code
>>>>>>>>> (most of
>>>>>>> them in the back-end, but also the front-end is big enough to
>>>>>>> not
>>>>>>> trace
>>>>>>> through every lane if I want to get to the point in this life
>>>>>>> :-D
>>>>>>> ).
>>>>>>>>> The application now runs for nearly 2 years on the
>>>>>>>>> customers
>>>>>>>>> site and
>>>>>>> is very reliable and fast in its daily usage. Apart from
>>>>>>> occasional updates
>>>>>>> every few month, it will never be restarted or need any
>>>>>>> manual
>>>>>>> maintenance
>>>>>>> at all. So I assume the code is not the badest on earth. At
>>>>>>> least
>>>>>>> because
>>>>>>> of that I'm able to out-rule things like memory leaks, gc
>>>>>>> issues
>>>>>>> or
>>>>>>> bottlenecks (max Heap is never reached, neither half of it,
>>>>>>> even
>>>>>>> after
>>>>>>> running for month).
>>>>>>>>> Of course there could be aspects of JSF which I may not
>>>>>>>>> have
>>>>>>>>> understood
>>>>>>> correctly until now. But because there are so many new things
>>>>>>> to
>>>>>>> a JSF
>>>>>>> beginner like me, I'm running out of ideas what this could
>>>>>>> be. It
>>>>>>> is
>>>>>>> something that worked well 'til TomEE 8.0.6 (Prod and Dev
>>>>>>> mode)
>>>>>>> and
>>>>>>> suddenly don't work anymore on 8.0.12 in Dev mode (but still
>>>>>>> works great in
>>>>>>> Prod mode). There are only minor versions in-between. So I
>>>>>>> think
>>>>>>> there were
>>>>>>> no revolutionary changes anywhere. My hope was (and still
>>>>>>> is),
>>>>>>> that anybody
>>>>>>> here on the list might have had a similar problem and has
>>>>>>> solved
>>>>>>> it, or
>>>>>>> maybe someone knows about some changes in Dev mode which
>>>>>>> could
>>>>>>> explain such
>>>>>>> behaviour (don't know, maybe xml parsing or component tree
>>>>>>> generation has
>>>>>>> changed, or scope handling of managed beans, or whatever
>>>>>>> could
>>>>>>> cause some
>>>>>>> code to be entered far more often than before, or some
>>>>>>> cache/pool
>>>>>>> was
>>>>>>> removed or has different behaviour... such things). Then it
>>>>>>> would
>>>>>>> be far
>>>>>>> more easy for me to track down the real problem.
>>>>>>>>> Anyway, when I get some spare time (currently have some
>>>>>>>>> deadlines in
>>>>>>> sight), I'll try to build a small test case. Either I can get
>>>>>>> a
>>>>>>> grip on the
>>>>>>> issue myself on doing this, or I can supply it to anyone who
>>>>>>> might want to
>>>>>>> have a look at it.
>>>>>>>>> cu
>>>>>>>>> Jens
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
>>>>>>>>>>> newer versions are so incredibly slow in dev mode
>>>>>>>>>> I can assure you that Dev/Prod mode works very very
>>>>>>>>>> swiftly
>>>>>>>>>> on 8.0.15
>>>>>>>>>> for sizable applications. For example, I have a giant
>>>>>>>>>> application
>>>>>>>>>> (thousands of LOC, 50+ jsf views) that has sub-8ms
>>>>>>>>>> response
>>>>>>>>>> times
>>>>>>>>>> (minus database time). More than likely they are doing
>>>>>>>>>> something they
>>>>>>>>>> should not be doing with JSF apis and are causing
>>>>>>>>>> problems.
>>>>>>>>>>
>>>>>>>>>> I would check the obvious things first with VisualVM:
>>>>>>>>>> heap
>>>>>>>>>> size, gc
>>>>>>>>>> frequency, etc. You can also run their profiler and see
>>>>>>>>>> if
>>>>>>>>>> you can
>>>>>>>>>> spot anything immediately obvious.
>>>>>>>>>>
>>>>>>>>>> After that, the easiest way to locate the code slowing
>>>>>>>>>> you
>>>>>>>>>> down is
>>>>>>>>>> bisection. Cut half the code out, run, and continue
>>>>>>>>>> cutting
>>>>>>>>>> until you
>>>>>>>>>> can locate the problematic code. Then after that,
>>>>>>>>>> isolate a
>>>>>>>>>> test case
>>>>>>>>>> where flipping a boolean flag triggers the issue and
>>>>>>>>>> post
>>>>>>>>>> the results
>>>>>>>>>> here.
>>>>>>>>>>
>>>>>>>>>> I would get ahold of: https://www.tomitribe.com who are
>>>>>>>>>> literal
>>>>>>>>>> experts in this stuff. They offer professional services
>>>>>>>>>> to
>>>>>>>>>> handle
>>>>>>>>>> exactly these situations and can work directly in your
>>>>>>>>>> codebase to
>>>>>>>>>> help you find and fix the problem.
>>>>>>>>>>
>>>>>>>>>> On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski
>>>>>>>>>> <jz...@diegurus.de> wrote:
>>>>>>>>>>> Hi altogether,
>>>>>>>>>>>
>>>>>>>>>>> I'm new to this list and hope it's the right place to
>>>>>>>>>>> ask
>>>>>>>>>>> this
>>>>>>> question.
>>>>>>>>>>> If not, and someone know the right place to ask this,
>>>>>>>>>>> please give
>>>>>>> advise.
>>>>>>>>>>> I'm developing a big JSF Application for a customer.
>>>>>>>>>>> It's
>>>>>>>>>>> a long
>>>>>>> running
>>>>>>>>>>> project and development and when I started with it,
>>>>>>>>>>> TomEE
>>>>>>>>>>> 8.0.6 was
>>>>>>> the
>>>>>>>>>>> most recent Version of TomEE. The switch to Jakarta
>>>>>>>>>>> is
>>>>>>>>>>> planned for
>>>>>>> next
>>>>>>>>>>> year, so I'm still on the 8.x path for the time being
>>>>>>>>>>> and
>>>>>>>>>>> wasn't able
>>>>>>> to
>>>>>>>>>>> test if this problem still exists in the 9.x branch.
>>>>>>>>>>>
>>>>>>>>>>> The problem:
>>>>>>>>>>> On the customers site I'm with the latest TomEE
>>>>>>>>>>> version
>>>>>>>>>>> 8.0.15
>>>>>>> (running
>>>>>>>>>>> on Java 11) and everything works fine, because it is
>>>>>>>>>>> running in
>>>>>>>>>>> production mode. But in my development environment
>>>>>>>>>>> I'm
>>>>>>>>>>> still stuck
>>>>>>> with
>>>>>>>>>>> 8.0.6, because there I need the development mode
>>>>>>>>>>> (javax.faces.PROJECT_STAGE: Development). I need to
>>>>>>>>>>> be
>>>>>>>>>>> able to see
>>>>>>>>>>> changes in facelets without restarting everything
>>>>>>>>>>> everytime, and to
>>>>>>> get
>>>>>>>>>>> more detailed error messages.
>>>>>>>>>>> My attempts to update my dev TomEE to something newer
>>>>>>>>>>> than 8.0.6 all
>>>>>>>>>>> failed, because all (at least all I've tested so far)
>>>>>>>>>>> newer versions
>>>>>>> are
>>>>>>>>>>> so incredibly slow in dev mode, that it's unbearable
>>>>>>>>>>> to
>>>>>>>>>>> use the
>>>>>>>>>>> application. Several very looong seconds on every
>>>>>>>>>>> request
>>>>>>>>>>> (even the
>>>>>>>>>>> little AJAX requests in a view) is simply not a
>>>>>>>>>>> practical
>>>>>>>>>>> environment.
>>>>>>>>>>> After switching to Production mode everything works
>>>>>>>>>>> very
>>>>>>>>>>> fast, and all
>>>>>>>>>>> my views have response times of very few ms (even the
>>>>>>>>>>> big
>>>>>>>>>>> ones with
>>>>>>> max.
>>>>>>>>>>> around 400ms). When switching back to Development
>>>>>>>>>>> mode I
>>>>>>>>>>> have response
>>>>>>>>>>> times of up to 30s on big views. Not with 8.0.6,
>>>>>>>>>>> there
>>>>>>>>>>> even in
>>>>>>>>>>> development mode it's reasonable fast in not getting
>>>>>>>>>>> higher than 1s.
>>>>>>>>>>>
>>>>>>>>>>> My question:
>>>>>>>>>>> What causes this enormous performance degradation?
>>>>>>>>>>> I'm
>>>>>>>>>>> hoping, it is
>>>>>>>>>>> just a configuration which now has another default
>>>>>>>>>>> value
>>>>>>>>>>> or the like.
>>>>>>> If
>>>>>>>>>>> yes, maybe someone can point me in the right
>>>>>>>>>>> direction of
>>>>>>>>>>> where to
>>>>>>> find
>>>>>>>>>>> this configuration? If it's not a configuration
>>>>>>>>>>> thing:
>>>>>>>>>>> what can I do
>>>>>>> to
>>>>>>>>>>> get around this?
>>>>>>>>>>>
>>>>>>>>>>> I haven't tested all Versions of TomEE from 8.0.6 to
>>>>>>>>>>> 8.0.15, so I
>>>>>>> can't
>>>>>>>>>>> say at what version exactly this behaviour changes.
>>>>>>>>>>> If it
>>>>>>>>>>> helps or is
>>>>>>>>>>> needed, I can make some tests to find the exact
>>>>>>>>>>> version
>>>>>>>>>>> where this
>>>>>>>>>>> happens. Versions I've already tested are: 8.0.12,
>>>>>>>>>>> 8.0.13
>>>>>>>>>>> and 8.0.15.
>>>>>>>>>>> All of them are very slow in dev mode. If you need
>>>>>>>>>>> more
>>>>>>>>>>> information,
>>>>>>>>>>> I'll try to provide it.
>>>>>>>>>>>
>>>>>>>>>>> Thanks in advance for any help
>>>>>>>>>>> cu
>>>>>>>>>>> Jens
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Jens Zurawski <jz...@diegurus.de>.
Thanks Richard (and Vicente),
I've changed to JDK 8 and now the builds are successful (both 8.0.6 and
8.0.8).
I'll script me some test scripts to ease the build, run and test for
every bisect step.
A build needs around 3 to 4 minutes. Running a test with my application
will need some manual interaction and checks, I guess also something
around 4 minutes. This sums up to roughly 8 minutes per step.
I've started a bisect and it told me:
git bisect good 20441eb
git bisect bad 4c8a616
Bisecting: 136 revisions left to test after this (roughly 7 steps)
That means, if everything works fine, I should be able to identify the
commit after about an hour? (That's cool.... why nobody has told me
about git bisect before? ;-) )
Anyway, I will have to find some time for this, maybe I need a couple of
days to find some.
I will let you know about the outcome.
cu
Jens
Am 27.10.2023 um 11:50 schrieb Richard Zowalla:
> Hi,
>
> you need Java/JDK 8 to build TomEE 8.
>
> The Maven command is
>
> mvn --fail-at-end clean install -pl tomee/apache-tomee -am -
> Dfile.encoding=UTF-8 -DskipTests=true
>
> You will find the tar.gz/zip in ./tomee/apache-tomee/target
>
> Gruß
> Richard
>
> P.S. 8.0.7 is here: https://archive.apache.org/dist/tomee/tomee-8.0.7/
>
> Am Freitag, dem 27.10.2023 um 10:50 +0200 schrieb Jens Zurawski:
>> Hi Richard,
>>
>> to identify the "bad" version was easy. I didn't find a 8.0.7 release
>> on
>> the download page, so I started with 8.0.8. And this version already
>> has
>> the slow dev mode. So for my problem I can say:
>> good: 8.0.6
>> bad: 8.0.8
>>
>> Now, I think, I need a little help from you. I tried to build a TomEE
>> out of git (I've cloned the github repository
>> https://github.com/apache/tomee.git). I've tried these two commits:
>> 20441eb (tag: tomee-8.0.6)
>> 4c8a616 (tag: tomee-project-8.0.8)
>>
>> But the build fails every time on "TomEE :: Container :: Core" with:
>>
>> [INFO] BUILD FAILURE
>> [INFO]
>> ---------------------------------------------------------------------
>> ---
>> [INFO] Total time: 01:19 min
>> [INFO] Finished at: 2023-10-27T10:29:40+02:00
>> [INFO]
>> ---------------------------------------------------------------------
>> ---
>> [ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile
>> (default-compile) on project openejb-core: Compilation failure:
>> Compilation failure:
>> [ERROR]
>> /sandbox/tomee/container/openejb-
>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[21,21]
>> package org.omg.CORBA does not exist
>> [ERROR]
>> /sandbox/tomee/container/openejb-
>> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[24,12]
>> cannot find symbol
>> [ERROR] symbol: class ORB
>> [ERROR] location: class org.apache.openejb.core.OrbFactory
>>
>> I've tried to build with:
>> mvn clean install
>>
>> My build environment is a linux ubuntu 20.04.6 LTS VM with:
>> git 2.25.1
>> Maven 3.6.3
>> java 14.0.2
>>
>> Is it some easy thing to fix, or do I have to switch some versions of
>> my
>> environment?
>>
>> cu
>> Jens
>>
>> Am 26.10.2023 um 19:48 schrieb Richard Zowalla:
>>> Hi,
>>>
>>> without knowing code or having a reproducer, it might be not that
>>> obvious to find a reason / answer for the "academic problem" ;-)
>>>
>>> If you really want to boil it down, I recommend the following
>>> approach:
>>>
>>> (1) From your description it seems, that 8.0.6 is the last known
>>> "good"
>>> version.
>>>
>>> (2) You've investigated versions starting from 8.0.12 onwards,
>>> which
>>> seems to be "bad".
>>>
>>> Following the approach recommended by Jonathan:
>>>
>>> - Can you check if it happens in 8.0.7 (8.0.8, ...) until we find
>>> the
>>> first "bad" released version?
>>>
>>> - If we have that, you could start a "git bisect" session between
>>> 8.0.6
>>> and the first version, which is known to be bad (reducing the
>>> amount of
>>> needed bisect cycles dramatically), so we find a commit / reason
>>> (if it
>>> is in TomEE).
>>>
>>> - It would involve some quick builds without executing tests (~10-
>>> 15min
>>> each) for each bisect step and testing on your side.
>>>
>>> Don't know if your inner monk is willing to go that route, but I am
>>> happy to help with the related Maven commands for quick building :)
>>>
>>> Gruß
>>> Richard
>>>
>>>
>>>
>>>
>>>
>>> Am Donnerstag, dem 26.10.2023 um 18:00 +0200 schrieb Jens Zurawski:
>>>> Hi Vicente,
>>>>
>>>> thank you for the tip. This is a good workaround. When I switch
>>>> to
>>>> Production mode and insert this configuration, it reloads the
>>>> view
>>>> from
>>>> the file if it has changed. This means, I can use a decent
>>>> version of
>>>> TomEE in my dev environment.
>>>>
>>>> However, it doesn't completely solve my problem, as the error
>>>> reporting
>>>> isn't as verbose as in Dev mode as far as I can see. But I have
>>>> to
>>>> check, maybe it's enough for my development.
>>>>
>>>> And, well.... the academic problem: "Why is it still broken in
>>>> Dev
>>>> mode?" still runs circles through my brain ;-) Because if it's a
>>>> problem
>>>> of my coding, it might be a good idea to change this, before it
>>>> might
>>>> get a problem on Production mode, too, at some time in the
>>>> future.
>>>>
>>>> But anyway, you've helped me a good step forward, thanks.
>>>> cu
>>>> Jens
>>>>
>>>>
>>>> Am 26.10.2023 um 17:04 schrieb Vicente Rossello:
>>>>> We are using the production stage with
>>>>> javax.faces.FACELETS_REFRESH_PERIOD
>>>>> = 1. It's faster and more similar to production, just try it if
>>>>> it
>>>>> solves
>>>>> your problem
>>>>>
>>>>> On Thu, Oct 26, 2023 at 4:18 PM Jens Zurawski <jz...@diegurus.de>
>>>>> wrote:
>>>>>
>>>>>> The one bundled with TomEE.
>>>>>> myfaces 2.3.7 on TomEE 8.0.6
>>>>>> and e.g. myfaces 2.3.9 on TomEE 8.0.12
>>>>>>
>>>>>> Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
>>>>>>> Which JSF lib are you using?
>>>>>>>
>>>>>>> Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens Zurawski
>>>>>>> <jzu@diegurus.de
>>>>>>> :
>>>>>>>> Thank you for your quick reply, Jonathan.
>>>>>>>>
>>>>>>>> It is good to know, that normally the Dev mode should
>>>>>>>> work as
>>>>>>>> fast as
>>>>>> before even with 8.0.15. So it obviously has something to do
>>>>>> with
>>>>>> my
>>>>>> environment, configuration and/or code.
>>>>>>>> And also thank you for the reference to tomitribe.
>>>>>>>> Unfortunately I'm
>>>>>> the only developer on this project, and it will be difficult
>>>>>> to
>>>>>> convince my
>>>>>> customer to an additional support subscription, because he on
>>>>>> production
>>>>>> doesn't have issues ;-) But, ok, that's my problem, and maybe
>>>>>> I'll consult
>>>>>> tomitribe in the future for other projects.
>>>>>>>> So, I'm afraid, for the near future I'm on my own to
>>>>>>>> solve
>>>>>>>> this
>>>>>> problem. Therefore any hints to restrict the places where I
>>>>>> should take a
>>>>>> look will be more than welcome.
>>>>>>>> I'm a senior programmer with 20 years of experience in
>>>>>>>> Java
>>>>>>>> programming
>>>>>> (SE and JSP). But I'm relative new to JSF/JavaEE/JakartaEE
>>>>>> and
>>>>>> sometimes I
>>>>>> have a hard time to understand which component of the whole
>>>>>> stack
>>>>>> is
>>>>>> responsible for what.
>>>>>>>> The project meanwhile has over a 1/4 million lines of
>>>>>>>> code
>>>>>>>> (most of
>>>>>> them in the back-end, but also the front-end is big enough to
>>>>>> not
>>>>>> trace
>>>>>> through every lane if I want to get to the point in this life
>>>>>> :-D
>>>>>> ).
>>>>>>>> The application now runs for nearly 2 years on the
>>>>>>>> customers
>>>>>>>> site and
>>>>>> is very reliable and fast in its daily usage. Apart from
>>>>>> occasional updates
>>>>>> every few month, it will never be restarted or need any
>>>>>> manual
>>>>>> maintenance
>>>>>> at all. So I assume the code is not the badest on earth. At
>>>>>> least
>>>>>> because
>>>>>> of that I'm able to out-rule things like memory leaks, gc
>>>>>> issues
>>>>>> or
>>>>>> bottlenecks (max Heap is never reached, neither half of it,
>>>>>> even
>>>>>> after
>>>>>> running for month).
>>>>>>>> Of course there could be aspects of JSF which I may not
>>>>>>>> have
>>>>>>>> understood
>>>>>> correctly until now. But because there are so many new things
>>>>>> to
>>>>>> a JSF
>>>>>> beginner like me, I'm running out of ideas what this could
>>>>>> be. It
>>>>>> is
>>>>>> something that worked well 'til TomEE 8.0.6 (Prod and Dev
>>>>>> mode)
>>>>>> and
>>>>>> suddenly don't work anymore on 8.0.12 in Dev mode (but still
>>>>>> works great in
>>>>>> Prod mode). There are only minor versions in-between. So I
>>>>>> think
>>>>>> there were
>>>>>> no revolutionary changes anywhere. My hope was (and still
>>>>>> is),
>>>>>> that anybody
>>>>>> here on the list might have had a similar problem and has
>>>>>> solved
>>>>>> it, or
>>>>>> maybe someone knows about some changes in Dev mode which
>>>>>> could
>>>>>> explain such
>>>>>> behaviour (don't know, maybe xml parsing or component tree
>>>>>> generation has
>>>>>> changed, or scope handling of managed beans, or whatever
>>>>>> could
>>>>>> cause some
>>>>>> code to be entered far more often than before, or some
>>>>>> cache/pool
>>>>>> was
>>>>>> removed or has different behaviour... such things). Then it
>>>>>> would
>>>>>> be far
>>>>>> more easy for me to track down the real problem.
>>>>>>>> Anyway, when I get some spare time (currently have some
>>>>>>>> deadlines in
>>>>>> sight), I'll try to build a small test case. Either I can get
>>>>>> a
>>>>>> grip on the
>>>>>> issue myself on doing this, or I can supply it to anyone who
>>>>>> might want to
>>>>>> have a look at it.
>>>>>>>> cu
>>>>>>>> Jens
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
>>>>>>>>>> newer versions are so incredibly slow in dev mode
>>>>>>>>> I can assure you that Dev/Prod mode works very very
>>>>>>>>> swiftly
>>>>>>>>> on 8.0.15
>>>>>>>>> for sizable applications. For example, I have a giant
>>>>>>>>> application
>>>>>>>>> (thousands of LOC, 50+ jsf views) that has sub-8ms
>>>>>>>>> response
>>>>>>>>> times
>>>>>>>>> (minus database time). More than likely they are doing
>>>>>>>>> something they
>>>>>>>>> should not be doing with JSF apis and are causing
>>>>>>>>> problems.
>>>>>>>>>
>>>>>>>>> I would check the obvious things first with VisualVM:
>>>>>>>>> heap
>>>>>>>>> size, gc
>>>>>>>>> frequency, etc. You can also run their profiler and see
>>>>>>>>> if
>>>>>>>>> you can
>>>>>>>>> spot anything immediately obvious.
>>>>>>>>>
>>>>>>>>> After that, the easiest way to locate the code slowing
>>>>>>>>> you
>>>>>>>>> down is
>>>>>>>>> bisection. Cut half the code out, run, and continue
>>>>>>>>> cutting
>>>>>>>>> until you
>>>>>>>>> can locate the problematic code. Then after that,
>>>>>>>>> isolate a
>>>>>>>>> test case
>>>>>>>>> where flipping a boolean flag triggers the issue and
>>>>>>>>> post
>>>>>>>>> the results
>>>>>>>>> here.
>>>>>>>>>
>>>>>>>>> I would get ahold of: https://www.tomitribe.com who are
>>>>>>>>> literal
>>>>>>>>> experts in this stuff. They offer professional services
>>>>>>>>> to
>>>>>>>>> handle
>>>>>>>>> exactly these situations and can work directly in your
>>>>>>>>> codebase to
>>>>>>>>> help you find and fix the problem.
>>>>>>>>>
>>>>>>>>> On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski
>>>>>>>>> <jz...@diegurus.de> wrote:
>>>>>>>>>> Hi altogether,
>>>>>>>>>>
>>>>>>>>>> I'm new to this list and hope it's the right place to
>>>>>>>>>> ask
>>>>>>>>>> this
>>>>>> question.
>>>>>>>>>> If not, and someone know the right place to ask this,
>>>>>>>>>> please give
>>>>>> advise.
>>>>>>>>>> I'm developing a big JSF Application for a customer.
>>>>>>>>>> It's
>>>>>>>>>> a long
>>>>>> running
>>>>>>>>>> project and development and when I started with it,
>>>>>>>>>> TomEE
>>>>>>>>>> 8.0.6 was
>>>>>> the
>>>>>>>>>> most recent Version of TomEE. The switch to Jakarta
>>>>>>>>>> is
>>>>>>>>>> planned for
>>>>>> next
>>>>>>>>>> year, so I'm still on the 8.x path for the time being
>>>>>>>>>> and
>>>>>>>>>> wasn't able
>>>>>> to
>>>>>>>>>> test if this problem still exists in the 9.x branch.
>>>>>>>>>>
>>>>>>>>>> The problem:
>>>>>>>>>> On the customers site I'm with the latest TomEE
>>>>>>>>>> version
>>>>>>>>>> 8.0.15
>>>>>> (running
>>>>>>>>>> on Java 11) and everything works fine, because it is
>>>>>>>>>> running in
>>>>>>>>>> production mode. But in my development environment
>>>>>>>>>> I'm
>>>>>>>>>> still stuck
>>>>>> with
>>>>>>>>>> 8.0.6, because there I need the development mode
>>>>>>>>>> (javax.faces.PROJECT_STAGE: Development). I need to
>>>>>>>>>> be
>>>>>>>>>> able to see
>>>>>>>>>> changes in facelets without restarting everything
>>>>>>>>>> everytime, and to
>>>>>> get
>>>>>>>>>> more detailed error messages.
>>>>>>>>>> My attempts to update my dev TomEE to something newer
>>>>>>>>>> than 8.0.6 all
>>>>>>>>>> failed, because all (at least all I've tested so far)
>>>>>>>>>> newer versions
>>>>>> are
>>>>>>>>>> so incredibly slow in dev mode, that it's unbearable
>>>>>>>>>> to
>>>>>>>>>> use the
>>>>>>>>>> application. Several very looong seconds on every
>>>>>>>>>> request
>>>>>>>>>> (even the
>>>>>>>>>> little AJAX requests in a view) is simply not a
>>>>>>>>>> practical
>>>>>>>>>> environment.
>>>>>>>>>> After switching to Production mode everything works
>>>>>>>>>> very
>>>>>>>>>> fast, and all
>>>>>>>>>> my views have response times of very few ms (even the
>>>>>>>>>> big
>>>>>>>>>> ones with
>>>>>> max.
>>>>>>>>>> around 400ms). When switching back to Development
>>>>>>>>>> mode I
>>>>>>>>>> have response
>>>>>>>>>> times of up to 30s on big views. Not with 8.0.6,
>>>>>>>>>> there
>>>>>>>>>> even in
>>>>>>>>>> development mode it's reasonable fast in not getting
>>>>>>>>>> higher than 1s.
>>>>>>>>>>
>>>>>>>>>> My question:
>>>>>>>>>> What causes this enormous performance degradation?
>>>>>>>>>> I'm
>>>>>>>>>> hoping, it is
>>>>>>>>>> just a configuration which now has another default
>>>>>>>>>> value
>>>>>>>>>> or the like.
>>>>>> If
>>>>>>>>>> yes, maybe someone can point me in the right
>>>>>>>>>> direction of
>>>>>>>>>> where to
>>>>>> find
>>>>>>>>>> this configuration? If it's not a configuration
>>>>>>>>>> thing:
>>>>>>>>>> what can I do
>>>>>> to
>>>>>>>>>> get around this?
>>>>>>>>>>
>>>>>>>>>> I haven't tested all Versions of TomEE from 8.0.6 to
>>>>>>>>>> 8.0.15, so I
>>>>>> can't
>>>>>>>>>> say at what version exactly this behaviour changes.
>>>>>>>>>> If it
>>>>>>>>>> helps or is
>>>>>>>>>> needed, I can make some tests to find the exact
>>>>>>>>>> version
>>>>>>>>>> where this
>>>>>>>>>> happens. Versions I've already tested are: 8.0.12,
>>>>>>>>>> 8.0.13
>>>>>>>>>> and 8.0.15.
>>>>>>>>>> All of them are very slow in dev mode. If you need
>>>>>>>>>> more
>>>>>>>>>> information,
>>>>>>>>>> I'll try to provide it.
>>>>>>>>>>
>>>>>>>>>> Thanks in advance for any help
>>>>>>>>>> cu
>>>>>>>>>> Jens
>>>>>>>>>>
>>>>>>>>>>
>>>>>>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Richard Zowalla <ri...@zowalla.com>.
Hi,
you need Java/JDK 8 to build TomEE 8.
The Maven command is
mvn --fail-at-end clean install -pl tomee/apache-tomee -am -
Dfile.encoding=UTF-8 -DskipTests=true
You will find the tar.gz/zip in ./tomee/apache-tomee/target
Gruß
Richard
P.S. 8.0.7 is here: https://archive.apache.org/dist/tomee/tomee-8.0.7/
Am Freitag, dem 27.10.2023 um 10:50 +0200 schrieb Jens Zurawski:
> Hi Richard,
>
> to identify the "bad" version was easy. I didn't find a 8.0.7 release
> on
> the download page, so I started with 8.0.8. And this version already
> has
> the slow dev mode. So for my problem I can say:
> good: 8.0.6
> bad: 8.0.8
>
> Now, I think, I need a little help from you. I tried to build a TomEE
> out of git (I've cloned the github repository
> https://github.com/apache/tomee.git). I've tried these two commits:
> 20441eb (tag: tomee-8.0.6)
> 4c8a616 (tag: tomee-project-8.0.8)
>
> But the build fails every time on "TomEE :: Container :: Core" with:
>
> [INFO] BUILD FAILURE
> [INFO]
> ---------------------------------------------------------------------
> ---
> [INFO] Total time: 01:19 min
> [INFO] Finished at: 2023-10-27T10:29:40+02:00
> [INFO]
> ---------------------------------------------------------------------
> ---
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile
> (default-compile) on project openejb-core: Compilation failure:
> Compilation failure:
> [ERROR]
> /sandbox/tomee/container/openejb-
> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[21,21]
> package org.omg.CORBA does not exist
> [ERROR]
> /sandbox/tomee/container/openejb-
> core/src/main/java/org/apache/openejb/core/OrbFactory.java:[24,12]
> cannot find symbol
> [ERROR] symbol: class ORB
> [ERROR] location: class org.apache.openejb.core.OrbFactory
>
> I've tried to build with:
> mvn clean install
>
> My build environment is a linux ubuntu 20.04.6 LTS VM with:
> git 2.25.1
> Maven 3.6.3
> java 14.0.2
>
> Is it some easy thing to fix, or do I have to switch some versions of
> my
> environment?
>
> cu
> Jens
>
> Am 26.10.2023 um 19:48 schrieb Richard Zowalla:
> > Hi,
> >
> > without knowing code or having a reproducer, it might be not that
> > obvious to find a reason / answer for the "academic problem" ;-)
> >
> > If you really want to boil it down, I recommend the following
> > approach:
> >
> > (1) From your description it seems, that 8.0.6 is the last known
> > "good"
> > version.
> >
> > (2) You've investigated versions starting from 8.0.12 onwards,
> > which
> > seems to be "bad".
> >
> > Following the approach recommended by Jonathan:
> >
> > - Can you check if it happens in 8.0.7 (8.0.8, ...) until we find
> > the
> > first "bad" released version?
> >
> > - If we have that, you could start a "git bisect" session between
> > 8.0.6
> > and the first version, which is known to be bad (reducing the
> > amount of
> > needed bisect cycles dramatically), so we find a commit / reason
> > (if it
> > is in TomEE).
> >
> > - It would involve some quick builds without executing tests (~10-
> > 15min
> > each) for each bisect step and testing on your side.
> >
> > Don't know if your inner monk is willing to go that route, but I am
> > happy to help with the related Maven commands for quick building :)
> >
> > Gruß
> > Richard
> >
> >
> >
> >
> >
> > Am Donnerstag, dem 26.10.2023 um 18:00 +0200 schrieb Jens Zurawski:
> > > Hi Vicente,
> > >
> > > thank you for the tip. This is a good workaround. When I switch
> > > to
> > > Production mode and insert this configuration, it reloads the
> > > view
> > > from
> > > the file if it has changed. This means, I can use a decent
> > > version of
> > > TomEE in my dev environment.
> > >
> > > However, it doesn't completely solve my problem, as the error
> > > reporting
> > > isn't as verbose as in Dev mode as far as I can see. But I have
> > > to
> > > check, maybe it's enough for my development.
> > >
> > > And, well.... the academic problem: "Why is it still broken in
> > > Dev
> > > mode?" still runs circles through my brain ;-) Because if it's a
> > > problem
> > > of my coding, it might be a good idea to change this, before it
> > > might
> > > get a problem on Production mode, too, at some time in the
> > > future.
> > >
> > > But anyway, you've helped me a good step forward, thanks.
> > > cu
> > > Jens
> > >
> > >
> > > Am 26.10.2023 um 17:04 schrieb Vicente Rossello:
> > > > We are using the production stage with
> > > > javax.faces.FACELETS_REFRESH_PERIOD
> > > > = 1. It's faster and more similar to production, just try it if
> > > > it
> > > > solves
> > > > your problem
> > > >
> > > > On Thu, Oct 26, 2023 at 4:18 PM Jens Zurawski <jz...@diegurus.de>
> > > > wrote:
> > > >
> > > > > The one bundled with TomEE.
> > > > > myfaces 2.3.7 on TomEE 8.0.6
> > > > > and e.g. myfaces 2.3.9 on TomEE 8.0.12
> > > > >
> > > > > Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
> > > > > > Which JSF lib are you using?
> > > > > >
> > > > > > Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens Zurawski
> > > > > > <jzu@diegurus.de
> > > > > > :
> > > > > > > Thank you for your quick reply, Jonathan.
> > > > > > >
> > > > > > > It is good to know, that normally the Dev mode should
> > > > > > > work as
> > > > > > > fast as
> > > > > before even with 8.0.15. So it obviously has something to do
> > > > > with
> > > > > my
> > > > > environment, configuration and/or code.
> > > > > > > And also thank you for the reference to tomitribe.
> > > > > > > Unfortunately I'm
> > > > > the only developer on this project, and it will be difficult
> > > > > to
> > > > > convince my
> > > > > customer to an additional support subscription, because he on
> > > > > production
> > > > > doesn't have issues ;-) But, ok, that's my problem, and maybe
> > > > > I'll consult
> > > > > tomitribe in the future for other projects.
> > > > > > > So, I'm afraid, for the near future I'm on my own to
> > > > > > > solve
> > > > > > > this
> > > > > problem. Therefore any hints to restrict the places where I
> > > > > should take a
> > > > > look will be more than welcome.
> > > > > > > I'm a senior programmer with 20 years of experience in
> > > > > > > Java
> > > > > > > programming
> > > > > (SE and JSP). But I'm relative new to JSF/JavaEE/JakartaEE
> > > > > and
> > > > > sometimes I
> > > > > have a hard time to understand which component of the whole
> > > > > stack
> > > > > is
> > > > > responsible for what.
> > > > > > > The project meanwhile has over a 1/4 million lines of
> > > > > > > code
> > > > > > > (most of
> > > > > them in the back-end, but also the front-end is big enough to
> > > > > not
> > > > > trace
> > > > > through every lane if I want to get to the point in this life
> > > > > :-D
> > > > > ).
> > > > > > > The application now runs for nearly 2 years on the
> > > > > > > customers
> > > > > > > site and
> > > > > is very reliable and fast in its daily usage. Apart from
> > > > > occasional updates
> > > > > every few month, it will never be restarted or need any
> > > > > manual
> > > > > maintenance
> > > > > at all. So I assume the code is not the badest on earth. At
> > > > > least
> > > > > because
> > > > > of that I'm able to out-rule things like memory leaks, gc
> > > > > issues
> > > > > or
> > > > > bottlenecks (max Heap is never reached, neither half of it,
> > > > > even
> > > > > after
> > > > > running for month).
> > > > > > > Of course there could be aspects of JSF which I may not
> > > > > > > have
> > > > > > > understood
> > > > > correctly until now. But because there are so many new things
> > > > > to
> > > > > a JSF
> > > > > beginner like me, I'm running out of ideas what this could
> > > > > be. It
> > > > > is
> > > > > something that worked well 'til TomEE 8.0.6 (Prod and Dev
> > > > > mode)
> > > > > and
> > > > > suddenly don't work anymore on 8.0.12 in Dev mode (but still
> > > > > works great in
> > > > > Prod mode). There are only minor versions in-between. So I
> > > > > think
> > > > > there were
> > > > > no revolutionary changes anywhere. My hope was (and still
> > > > > is),
> > > > > that anybody
> > > > > here on the list might have had a similar problem and has
> > > > > solved
> > > > > it, or
> > > > > maybe someone knows about some changes in Dev mode which
> > > > > could
> > > > > explain such
> > > > > behaviour (don't know, maybe xml parsing or component tree
> > > > > generation has
> > > > > changed, or scope handling of managed beans, or whatever
> > > > > could
> > > > > cause some
> > > > > code to be entered far more often than before, or some
> > > > > cache/pool
> > > > > was
> > > > > removed or has different behaviour... such things). Then it
> > > > > would
> > > > > be far
> > > > > more easy for me to track down the real problem.
> > > > > > > Anyway, when I get some spare time (currently have some
> > > > > > > deadlines in
> > > > > sight), I'll try to build a small test case. Either I can get
> > > > > a
> > > > > grip on the
> > > > > issue myself on doing this, or I can supply it to anyone who
> > > > > might want to
> > > > > have a look at it.
> > > > > > > cu
> > > > > > > Jens
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
> > > > > > > > > newer versions are so incredibly slow in dev mode
> > > > > > > > I can assure you that Dev/Prod mode works very very
> > > > > > > > swiftly
> > > > > > > > on 8.0.15
> > > > > > > > for sizable applications. For example, I have a giant
> > > > > > > > application
> > > > > > > > (thousands of LOC, 50+ jsf views) that has sub-8ms
> > > > > > > > response
> > > > > > > > times
> > > > > > > > (minus database time). More than likely they are doing
> > > > > > > > something they
> > > > > > > > should not be doing with JSF apis and are causing
> > > > > > > > problems.
> > > > > > > >
> > > > > > > > I would check the obvious things first with VisualVM:
> > > > > > > > heap
> > > > > > > > size, gc
> > > > > > > > frequency, etc. You can also run their profiler and see
> > > > > > > > if
> > > > > > > > you can
> > > > > > > > spot anything immediately obvious.
> > > > > > > >
> > > > > > > > After that, the easiest way to locate the code slowing
> > > > > > > > you
> > > > > > > > down is
> > > > > > > > bisection. Cut half the code out, run, and continue
> > > > > > > > cutting
> > > > > > > > until you
> > > > > > > > can locate the problematic code. Then after that,
> > > > > > > > isolate a
> > > > > > > > test case
> > > > > > > > where flipping a boolean flag triggers the issue and
> > > > > > > > post
> > > > > > > > the results
> > > > > > > > here.
> > > > > > > >
> > > > > > > > I would get ahold of: https://www.tomitribe.com who are
> > > > > > > > literal
> > > > > > > > experts in this stuff. They offer professional services
> > > > > > > > to
> > > > > > > > handle
> > > > > > > > exactly these situations and can work directly in your
> > > > > > > > codebase to
> > > > > > > > help you find and fix the problem.
> > > > > > > >
> > > > > > > > On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski
> > > > > > > > <jz...@diegurus.de> wrote:
> > > > > > > > > Hi altogether,
> > > > > > > > >
> > > > > > > > > I'm new to this list and hope it's the right place to
> > > > > > > > > ask
> > > > > > > > > this
> > > > > question.
> > > > > > > > > If not, and someone know the right place to ask this,
> > > > > > > > > please give
> > > > > advise.
> > > > > > > > > I'm developing a big JSF Application for a customer.
> > > > > > > > > It's
> > > > > > > > > a long
> > > > > running
> > > > > > > > > project and development and when I started with it,
> > > > > > > > > TomEE
> > > > > > > > > 8.0.6 was
> > > > > the
> > > > > > > > > most recent Version of TomEE. The switch to Jakarta
> > > > > > > > > is
> > > > > > > > > planned for
> > > > > next
> > > > > > > > > year, so I'm still on the 8.x path for the time being
> > > > > > > > > and
> > > > > > > > > wasn't able
> > > > > to
> > > > > > > > > test if this problem still exists in the 9.x branch.
> > > > > > > > >
> > > > > > > > > The problem:
> > > > > > > > > On the customers site I'm with the latest TomEE
> > > > > > > > > version
> > > > > > > > > 8.0.15
> > > > > (running
> > > > > > > > > on Java 11) and everything works fine, because it is
> > > > > > > > > running in
> > > > > > > > > production mode. But in my development environment
> > > > > > > > > I'm
> > > > > > > > > still stuck
> > > > > with
> > > > > > > > > 8.0.6, because there I need the development mode
> > > > > > > > > (javax.faces.PROJECT_STAGE: Development). I need to
> > > > > > > > > be
> > > > > > > > > able to see
> > > > > > > > > changes in facelets without restarting everything
> > > > > > > > > everytime, and to
> > > > > get
> > > > > > > > > more detailed error messages.
> > > > > > > > > My attempts to update my dev TomEE to something newer
> > > > > > > > > than 8.0.6 all
> > > > > > > > > failed, because all (at least all I've tested so far)
> > > > > > > > > newer versions
> > > > > are
> > > > > > > > > so incredibly slow in dev mode, that it's unbearable
> > > > > > > > > to
> > > > > > > > > use the
> > > > > > > > > application. Several very looong seconds on every
> > > > > > > > > request
> > > > > > > > > (even the
> > > > > > > > > little AJAX requests in a view) is simply not a
> > > > > > > > > practical
> > > > > > > > > environment.
> > > > > > > > > After switching to Production mode everything works
> > > > > > > > > very
> > > > > > > > > fast, and all
> > > > > > > > > my views have response times of very few ms (even the
> > > > > > > > > big
> > > > > > > > > ones with
> > > > > max.
> > > > > > > > > around 400ms). When switching back to Development
> > > > > > > > > mode I
> > > > > > > > > have response
> > > > > > > > > times of up to 30s on big views. Not with 8.0.6,
> > > > > > > > > there
> > > > > > > > > even in
> > > > > > > > > development mode it's reasonable fast in not getting
> > > > > > > > > higher than 1s.
> > > > > > > > >
> > > > > > > > > My question:
> > > > > > > > > What causes this enormous performance degradation?
> > > > > > > > > I'm
> > > > > > > > > hoping, it is
> > > > > > > > > just a configuration which now has another default
> > > > > > > > > value
> > > > > > > > > or the like.
> > > > > If
> > > > > > > > > yes, maybe someone can point me in the right
> > > > > > > > > direction of
> > > > > > > > > where to
> > > > > find
> > > > > > > > > this configuration? If it's not a configuration
> > > > > > > > > thing:
> > > > > > > > > what can I do
> > > > > to
> > > > > > > > > get around this?
> > > > > > > > >
> > > > > > > > > I haven't tested all Versions of TomEE from 8.0.6 to
> > > > > > > > > 8.0.15, so I
> > > > > can't
> > > > > > > > > say at what version exactly this behaviour changes.
> > > > > > > > > If it
> > > > > > > > > helps or is
> > > > > > > > > needed, I can make some tests to find the exact
> > > > > > > > > version
> > > > > > > > > where this
> > > > > > > > > happens. Versions I've already tested are: 8.0.12,
> > > > > > > > > 8.0.13
> > > > > > > > > and 8.0.15.
> > > > > > > > > All of them are very slow in dev mode. If you need
> > > > > > > > > more
> > > > > > > > > information,
> > > > > > > > > I'll try to provide it.
> > > > > > > > >
> > > > > > > > > Thanks in advance for any help
> > > > > > > > > cu
> > > > > > > > > Jens
> > > > > > > > >
> > > > > > > > >
> > > > > --
> > > > > jens zurawski
> > > > > diegurus - zurawski zurawski poppl rohland GbR
> > > > > juister straße 3
> > > > > 65199 wiesbaden
> > > > >
> > > > > kaspersweg 7b
> > > > > 26131 oldenburg
> > > > >
> > > > > internet http://www.diegurus.de
> > > > >
> > > > > tel +49(0)611 72437966
> > > > >
> > > > >
> > > > > CONFIDENTIALITY NOTICE: This e-mail message is intended only
> > > > > for
> > > > > the
> > > > > person or entity to which it is addressed and may contain
> > > > > confidential
> > > > > and/or privileged material. Any unauthorized review, use,
> > > > > disclosure or
> > > > > distribution is prohibited. If you are not the intended
> > > > > recipient,
> > > > > please contact the sender by reply e-mail and destroy all
> > > > > copies
> > > > > of the
> > > > > original message. If you are the intended recipient but do
> > > > > not
> > > > > wish to
> > > > > receive communications through this medium, please so advise
> > > > > the
> > > > > sender
> > > > > immediately.
> > > > >
> > > > >
>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Vicente Rossello <co...@gmail.com>.
You need java 8 to compile tomee 8. Corba packages were removed in jdk 11.
Try it
On Fri, Oct 27, 2023 at 10:51 AM Jens Zurawski <jz...@diegurus.de> wrote:
> Hi Richard,
>
> to identify the "bad" version was easy. I didn't find a 8.0.7 release on
> the download page, so I started with 8.0.8. And this version already has
> the slow dev mode. So for my problem I can say:
> good: 8.0.6
> bad: 8.0.8
>
> Now, I think, I need a little help from you. I tried to build a TomEE
> out of git (I've cloned the github repository
> https://github.com/apache/tomee.git). I've tried these two commits:
> 20441eb (tag: tomee-8.0.6)
> 4c8a616 (tag: tomee-project-8.0.8)
>
> But the build fails every time on "TomEE :: Container :: Core" with:
>
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 01:19 min
> [INFO] Finished at: 2023-10-27T10:29:40+02:00
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile
> (default-compile) on project openejb-core: Compilation failure:
> Compilation failure:
> [ERROR]
> /sandbox/tomee/container/openejb-core/src/main/java/org/apache/openejb/core/OrbFactory.java:[21,21]
>
> package org.omg.CORBA does not exist
> [ERROR]
> /sandbox/tomee/container/openejb-core/src/main/java/org/apache/openejb/core/OrbFactory.java:[24,12]
>
> cannot find symbol
> [ERROR] symbol: class ORB
> [ERROR] location: class org.apache.openejb.core.OrbFactory
>
> I've tried to build with:
> mvn clean install
>
> My build environment is a linux ubuntu 20.04.6 LTS VM with:
> git 2.25.1
> Maven 3.6.3
> java 14.0.2
>
> Is it some easy thing to fix, or do I have to switch some versions of my
> environment?
>
> cu
> Jens
>
> Am 26.10.2023 um 19:48 schrieb Richard Zowalla:
> > Hi,
> >
> > without knowing code or having a reproducer, it might be not that
> > obvious to find a reason / answer for the "academic problem" ;-)
> >
> > If you really want to boil it down, I recommend the following approach:
> >
> > (1) From your description it seems, that 8.0.6 is the last known "good"
> > version.
> >
> > (2) You've investigated versions starting from 8.0.12 onwards, which
> > seems to be "bad".
> >
> > Following the approach recommended by Jonathan:
> >
> > - Can you check if it happens in 8.0.7 (8.0.8, ...) until we find the
> > first "bad" released version?
> >
> > - If we have that, you could start a "git bisect" session between 8.0.6
> > and the first version, which is known to be bad (reducing the amount of
> > needed bisect cycles dramatically), so we find a commit / reason (if it
> > is in TomEE).
> >
> > - It would involve some quick builds without executing tests (~10-15min
> > each) for each bisect step and testing on your side.
> >
> > Don't know if your inner monk is willing to go that route, but I am
> > happy to help with the related Maven commands for quick building :)
> >
> > Gruß
> > Richard
> >
> >
> >
> >
> >
> > Am Donnerstag, dem 26.10.2023 um 18:00 +0200 schrieb Jens Zurawski:
> >> Hi Vicente,
> >>
> >> thank you for the tip. This is a good workaround. When I switch to
> >> Production mode and insert this configuration, it reloads the view
> >> from
> >> the file if it has changed. This means, I can use a decent version of
> >> TomEE in my dev environment.
> >>
> >> However, it doesn't completely solve my problem, as the error
> >> reporting
> >> isn't as verbose as in Dev mode as far as I can see. But I have to
> >> check, maybe it's enough for my development.
> >>
> >> And, well.... the academic problem: "Why is it still broken in Dev
> >> mode?" still runs circles through my brain ;-) Because if it's a
> >> problem
> >> of my coding, it might be a good idea to change this, before it might
> >> get a problem on Production mode, too, at some time in the future.
> >>
> >> But anyway, you've helped me a good step forward, thanks.
> >> cu
> >> Jens
> >>
> >>
> >> Am 26.10.2023 um 17:04 schrieb Vicente Rossello:
> >>> We are using the production stage with
> >>> javax.faces.FACELETS_REFRESH_PERIOD
> >>> = 1. It's faster and more similar to production, just try it if it
> >>> solves
> >>> your problem
> >>>
> >>> On Thu, Oct 26, 2023 at 4:18 PM Jens Zurawski <jz...@diegurus.de>
> >>> wrote:
> >>>
> >>>> The one bundled with TomEE.
> >>>> myfaces 2.3.7 on TomEE 8.0.6
> >>>> and e.g. myfaces 2.3.9 on TomEE 8.0.12
> >>>>
> >>>> Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
> >>>>> Which JSF lib are you using?
> >>>>>
> >>>>> Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens Zurawski
> >>>>> <jzu@diegurus.de
> >>>>> :
> >>>>>> Thank you for your quick reply, Jonathan.
> >>>>>>
> >>>>>> It is good to know, that normally the Dev mode should work as
> >>>>>> fast as
> >>>> before even with 8.0.15. So it obviously has something to do with
> >>>> my
> >>>> environment, configuration and/or code.
> >>>>>> And also thank you for the reference to tomitribe.
> >>>>>> Unfortunately I'm
> >>>> the only developer on this project, and it will be difficult to
> >>>> convince my
> >>>> customer to an additional support subscription, because he on
> >>>> production
> >>>> doesn't have issues ;-) But, ok, that's my problem, and maybe
> >>>> I'll consult
> >>>> tomitribe in the future for other projects.
> >>>>>> So, I'm afraid, for the near future I'm on my own to solve
> >>>>>> this
> >>>> problem. Therefore any hints to restrict the places where I
> >>>> should take a
> >>>> look will be more than welcome.
> >>>>>> I'm a senior programmer with 20 years of experience in Java
> >>>>>> programming
> >>>> (SE and JSP). But I'm relative new to JSF/JavaEE/JakartaEE and
> >>>> sometimes I
> >>>> have a hard time to understand which component of the whole stack
> >>>> is
> >>>> responsible for what.
> >>>>>> The project meanwhile has over a 1/4 million lines of code
> >>>>>> (most of
> >>>> them in the back-end, but also the front-end is big enough to not
> >>>> trace
> >>>> through every lane if I want to get to the point in this life :-D
> >>>> ).
> >>>>>> The application now runs for nearly 2 years on the customers
> >>>>>> site and
> >>>> is very reliable and fast in its daily usage. Apart from
> >>>> occasional updates
> >>>> every few month, it will never be restarted or need any manual
> >>>> maintenance
> >>>> at all. So I assume the code is not the badest on earth. At least
> >>>> because
> >>>> of that I'm able to out-rule things like memory leaks, gc issues
> >>>> or
> >>>> bottlenecks (max Heap is never reached, neither half of it, even
> >>>> after
> >>>> running for month).
> >>>>>> Of course there could be aspects of JSF which I may not have
> >>>>>> understood
> >>>> correctly until now. But because there are so many new things to
> >>>> a JSF
> >>>> beginner like me, I'm running out of ideas what this could be. It
> >>>> is
> >>>> something that worked well 'til TomEE 8.0.6 (Prod and Dev mode)
> >>>> and
> >>>> suddenly don't work anymore on 8.0.12 in Dev mode (but still
> >>>> works great in
> >>>> Prod mode). There are only minor versions in-between. So I think
> >>>> there were
> >>>> no revolutionary changes anywhere. My hope was (and still is),
> >>>> that anybody
> >>>> here on the list might have had a similar problem and has solved
> >>>> it, or
> >>>> maybe someone knows about some changes in Dev mode which could
> >>>> explain such
> >>>> behaviour (don't know, maybe xml parsing or component tree
> >>>> generation has
> >>>> changed, or scope handling of managed beans, or whatever could
> >>>> cause some
> >>>> code to be entered far more often than before, or some cache/pool
> >>>> was
> >>>> removed or has different behaviour... such things). Then it would
> >>>> be far
> >>>> more easy for me to track down the real problem.
> >>>>>> Anyway, when I get some spare time (currently have some
> >>>>>> deadlines in
> >>>> sight), I'll try to build a small test case. Either I can get a
> >>>> grip on the
> >>>> issue myself on doing this, or I can supply it to anyone who
> >>>> might want to
> >>>> have a look at it.
> >>>>>> cu
> >>>>>> Jens
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
> >>>>>>>> newer versions are so incredibly slow in dev mode
> >>>>>>> I can assure you that Dev/Prod mode works very very swiftly
> >>>>>>> on 8.0.15
> >>>>>>> for sizable applications. For example, I have a giant
> >>>>>>> application
> >>>>>>> (thousands of LOC, 50+ jsf views) that has sub-8ms response
> >>>>>>> times
> >>>>>>> (minus database time). More than likely they are doing
> >>>>>>> something they
> >>>>>>> should not be doing with JSF apis and are causing problems.
> >>>>>>>
> >>>>>>> I would check the obvious things first with VisualVM: heap
> >>>>>>> size, gc
> >>>>>>> frequency, etc. You can also run their profiler and see if
> >>>>>>> you can
> >>>>>>> spot anything immediately obvious.
> >>>>>>>
> >>>>>>> After that, the easiest way to locate the code slowing you
> >>>>>>> down is
> >>>>>>> bisection. Cut half the code out, run, and continue cutting
> >>>>>>> until you
> >>>>>>> can locate the problematic code. Then after that, isolate a
> >>>>>>> test case
> >>>>>>> where flipping a boolean flag triggers the issue and post
> >>>>>>> the results
> >>>>>>> here.
> >>>>>>>
> >>>>>>> I would get ahold of: https://www.tomitribe.com who are
> >>>>>>> literal
> >>>>>>> experts in this stuff. They offer professional services to
> >>>>>>> handle
> >>>>>>> exactly these situations and can work directly in your
> >>>>>>> codebase to
> >>>>>>> help you find and fix the problem.
> >>>>>>>
> >>>>>>> On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski
> >>>>>>> <jz...@diegurus.de> wrote:
> >>>>>>>> Hi altogether,
> >>>>>>>>
> >>>>>>>> I'm new to this list and hope it's the right place to ask
> >>>>>>>> this
> >>>> question.
> >>>>>>>> If not, and someone know the right place to ask this,
> >>>>>>>> please give
> >>>> advise.
> >>>>>>>> I'm developing a big JSF Application for a customer. It's
> >>>>>>>> a long
> >>>> running
> >>>>>>>> project and development and when I started with it, TomEE
> >>>>>>>> 8.0.6 was
> >>>> the
> >>>>>>>> most recent Version of TomEE. The switch to Jakarta is
> >>>>>>>> planned for
> >>>> next
> >>>>>>>> year, so I'm still on the 8.x path for the time being and
> >>>>>>>> wasn't able
> >>>> to
> >>>>>>>> test if this problem still exists in the 9.x branch.
> >>>>>>>>
> >>>>>>>> The problem:
> >>>>>>>> On the customers site I'm with the latest TomEE version
> >>>>>>>> 8.0.15
> >>>> (running
> >>>>>>>> on Java 11) and everything works fine, because it is
> >>>>>>>> running in
> >>>>>>>> production mode. But in my development environment I'm
> >>>>>>>> still stuck
> >>>> with
> >>>>>>>> 8.0.6, because there I need the development mode
> >>>>>>>> (javax.faces.PROJECT_STAGE: Development). I need to be
> >>>>>>>> able to see
> >>>>>>>> changes in facelets without restarting everything
> >>>>>>>> everytime, and to
> >>>> get
> >>>>>>>> more detailed error messages.
> >>>>>>>> My attempts to update my dev TomEE to something newer
> >>>>>>>> than 8.0.6 all
> >>>>>>>> failed, because all (at least all I've tested so far)
> >>>>>>>> newer versions
> >>>> are
> >>>>>>>> so incredibly slow in dev mode, that it's unbearable to
> >>>>>>>> use the
> >>>>>>>> application. Several very looong seconds on every request
> >>>>>>>> (even the
> >>>>>>>> little AJAX requests in a view) is simply not a practical
> >>>>>>>> environment.
> >>>>>>>> After switching to Production mode everything works very
> >>>>>>>> fast, and all
> >>>>>>>> my views have response times of very few ms (even the big
> >>>>>>>> ones with
> >>>> max.
> >>>>>>>> around 400ms). When switching back to Development mode I
> >>>>>>>> have response
> >>>>>>>> times of up to 30s on big views. Not with 8.0.6, there
> >>>>>>>> even in
> >>>>>>>> development mode it's reasonable fast in not getting
> >>>>>>>> higher than 1s.
> >>>>>>>>
> >>>>>>>> My question:
> >>>>>>>> What causes this enormous performance degradation? I'm
> >>>>>>>> hoping, it is
> >>>>>>>> just a configuration which now has another default value
> >>>>>>>> or the like.
> >>>> If
> >>>>>>>> yes, maybe someone can point me in the right direction of
> >>>>>>>> where to
> >>>> find
> >>>>>>>> this configuration? If it's not a configuration thing:
> >>>>>>>> what can I do
> >>>> to
> >>>>>>>> get around this?
> >>>>>>>>
> >>>>>>>> I haven't tested all Versions of TomEE from 8.0.6 to
> >>>>>>>> 8.0.15, so I
> >>>> can't
> >>>>>>>> say at what version exactly this behaviour changes. If it
> >>>>>>>> helps or is
> >>>>>>>> needed, I can make some tests to find the exact version
> >>>>>>>> where this
> >>>>>>>> happens. Versions I've already tested are: 8.0.12, 8.0.13
> >>>>>>>> and 8.0.15.
> >>>>>>>> All of them are very slow in dev mode. If you need more
> >>>>>>>> information,
> >>>>>>>> I'll try to provide it.
> >>>>>>>>
> >>>>>>>> Thanks in advance for any help
> >>>>>>>> cu
> >>>>>>>> Jens
> >>>>>>>>
> >>>>>>>>
> >>>> --
> >>>> jens zurawski
> >>>> diegurus - zurawski zurawski poppl rohland GbR
> >>>> juister straße 3
> >>>> 65199 wiesbaden
> >>>>
> >>>> kaspersweg 7b
> >>>> 26131 oldenburg
> >>>>
> >>>> internet http://www.diegurus.de
> >>>>
> >>>> tel +49(0)611 72437966
> >>>>
> >>>>
> >>>> CONFIDENTIALITY NOTICE: This e-mail message is intended only for
> >>>> the
> >>>> person or entity to which it is addressed and may contain
> >>>> confidential
> >>>> and/or privileged material. Any unauthorized review, use,
> >>>> disclosure or
> >>>> distribution is prohibited. If you are not the intended
> >>>> recipient,
> >>>> please contact the sender by reply e-mail and destroy all copies
> >>>> of the
> >>>> original message. If you are the intended recipient but do not
> >>>> wish to
> >>>> receive communications through this medium, please so advise the
> >>>> sender
> >>>> immediately.
> >>>>
> >>>>
>
>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Jens Zurawski <jz...@diegurus.de>.
Hi Richard,
to identify the "bad" version was easy. I didn't find a 8.0.7 release on
the download page, so I started with 8.0.8. And this version already has
the slow dev mode. So for my problem I can say:
good: 8.0.6
bad: 8.0.8
Now, I think, I need a little help from you. I tried to build a TomEE
out of git (I've cloned the github repository
https://github.com/apache/tomee.git). I've tried these two commits:
20441eb (tag: tomee-8.0.6)
4c8a616 (tag: tomee-project-8.0.8)
But the build fails every time on "TomEE :: Container :: Core" with:
[INFO] BUILD FAILURE
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 01:19 min
[INFO] Finished at: 2023-10-27T10:29:40+02:00
[INFO]
------------------------------------------------------------------------
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile
(default-compile) on project openejb-core: Compilation failure:
Compilation failure:
[ERROR]
/sandbox/tomee/container/openejb-core/src/main/java/org/apache/openejb/core/OrbFactory.java:[21,21]
package org.omg.CORBA does not exist
[ERROR]
/sandbox/tomee/container/openejb-core/src/main/java/org/apache/openejb/core/OrbFactory.java:[24,12]
cannot find symbol
[ERROR] symbol: class ORB
[ERROR] location: class org.apache.openejb.core.OrbFactory
I've tried to build with:
mvn clean install
My build environment is a linux ubuntu 20.04.6 LTS VM with:
git 2.25.1
Maven 3.6.3
java 14.0.2
Is it some easy thing to fix, or do I have to switch some versions of my
environment?
cu
Jens
Am 26.10.2023 um 19:48 schrieb Richard Zowalla:
> Hi,
>
> without knowing code or having a reproducer, it might be not that
> obvious to find a reason / answer for the "academic problem" ;-)
>
> If you really want to boil it down, I recommend the following approach:
>
> (1) From your description it seems, that 8.0.6 is the last known "good"
> version.
>
> (2) You've investigated versions starting from 8.0.12 onwards, which
> seems to be "bad".
>
> Following the approach recommended by Jonathan:
>
> - Can you check if it happens in 8.0.7 (8.0.8, ...) until we find the
> first "bad" released version?
>
> - If we have that, you could start a "git bisect" session between 8.0.6
> and the first version, which is known to be bad (reducing the amount of
> needed bisect cycles dramatically), so we find a commit / reason (if it
> is in TomEE).
>
> - It would involve some quick builds without executing tests (~10-15min
> each) for each bisect step and testing on your side.
>
> Don't know if your inner monk is willing to go that route, but I am
> happy to help with the related Maven commands for quick building :)
>
> Gruß
> Richard
>
>
>
>
>
> Am Donnerstag, dem 26.10.2023 um 18:00 +0200 schrieb Jens Zurawski:
>> Hi Vicente,
>>
>> thank you for the tip. This is a good workaround. When I switch to
>> Production mode and insert this configuration, it reloads the view
>> from
>> the file if it has changed. This means, I can use a decent version of
>> TomEE in my dev environment.
>>
>> However, it doesn't completely solve my problem, as the error
>> reporting
>> isn't as verbose as in Dev mode as far as I can see. But I have to
>> check, maybe it's enough for my development.
>>
>> And, well.... the academic problem: "Why is it still broken in Dev
>> mode?" still runs circles through my brain ;-) Because if it's a
>> problem
>> of my coding, it might be a good idea to change this, before it might
>> get a problem on Production mode, too, at some time in the future.
>>
>> But anyway, you've helped me a good step forward, thanks.
>> cu
>> Jens
>>
>>
>> Am 26.10.2023 um 17:04 schrieb Vicente Rossello:
>>> We are using the production stage with
>>> javax.faces.FACELETS_REFRESH_PERIOD
>>> = 1. It's faster and more similar to production, just try it if it
>>> solves
>>> your problem
>>>
>>> On Thu, Oct 26, 2023 at 4:18 PM Jens Zurawski <jz...@diegurus.de>
>>> wrote:
>>>
>>>> The one bundled with TomEE.
>>>> myfaces 2.3.7 on TomEE 8.0.6
>>>> and e.g. myfaces 2.3.9 on TomEE 8.0.12
>>>>
>>>> Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
>>>>> Which JSF lib are you using?
>>>>>
>>>>> Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens Zurawski
>>>>> <jzu@diegurus.de
>>>>> :
>>>>>> Thank you for your quick reply, Jonathan.
>>>>>>
>>>>>> It is good to know, that normally the Dev mode should work as
>>>>>> fast as
>>>> before even with 8.0.15. So it obviously has something to do with
>>>> my
>>>> environment, configuration and/or code.
>>>>>> And also thank you for the reference to tomitribe.
>>>>>> Unfortunately I'm
>>>> the only developer on this project, and it will be difficult to
>>>> convince my
>>>> customer to an additional support subscription, because he on
>>>> production
>>>> doesn't have issues ;-) But, ok, that's my problem, and maybe
>>>> I'll consult
>>>> tomitribe in the future for other projects.
>>>>>> So, I'm afraid, for the near future I'm on my own to solve
>>>>>> this
>>>> problem. Therefore any hints to restrict the places where I
>>>> should take a
>>>> look will be more than welcome.
>>>>>> I'm a senior programmer with 20 years of experience in Java
>>>>>> programming
>>>> (SE and JSP). But I'm relative new to JSF/JavaEE/JakartaEE and
>>>> sometimes I
>>>> have a hard time to understand which component of the whole stack
>>>> is
>>>> responsible for what.
>>>>>> The project meanwhile has over a 1/4 million lines of code
>>>>>> (most of
>>>> them in the back-end, but also the front-end is big enough to not
>>>> trace
>>>> through every lane if I want to get to the point in this life :-D
>>>> ).
>>>>>> The application now runs for nearly 2 years on the customers
>>>>>> site and
>>>> is very reliable and fast in its daily usage. Apart from
>>>> occasional updates
>>>> every few month, it will never be restarted or need any manual
>>>> maintenance
>>>> at all. So I assume the code is not the badest on earth. At least
>>>> because
>>>> of that I'm able to out-rule things like memory leaks, gc issues
>>>> or
>>>> bottlenecks (max Heap is never reached, neither half of it, even
>>>> after
>>>> running for month).
>>>>>> Of course there could be aspects of JSF which I may not have
>>>>>> understood
>>>> correctly until now. But because there are so many new things to
>>>> a JSF
>>>> beginner like me, I'm running out of ideas what this could be. It
>>>> is
>>>> something that worked well 'til TomEE 8.0.6 (Prod and Dev mode)
>>>> and
>>>> suddenly don't work anymore on 8.0.12 in Dev mode (but still
>>>> works great in
>>>> Prod mode). There are only minor versions in-between. So I think
>>>> there were
>>>> no revolutionary changes anywhere. My hope was (and still is),
>>>> that anybody
>>>> here on the list might have had a similar problem and has solved
>>>> it, or
>>>> maybe someone knows about some changes in Dev mode which could
>>>> explain such
>>>> behaviour (don't know, maybe xml parsing or component tree
>>>> generation has
>>>> changed, or scope handling of managed beans, or whatever could
>>>> cause some
>>>> code to be entered far more often than before, or some cache/pool
>>>> was
>>>> removed or has different behaviour... such things). Then it would
>>>> be far
>>>> more easy for me to track down the real problem.
>>>>>> Anyway, when I get some spare time (currently have some
>>>>>> deadlines in
>>>> sight), I'll try to build a small test case. Either I can get a
>>>> grip on the
>>>> issue myself on doing this, or I can supply it to anyone who
>>>> might want to
>>>> have a look at it.
>>>>>> cu
>>>>>> Jens
>>>>>>
>>>>>>
>>>>>>
>>>>>> Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
>>>>>>>> newer versions are so incredibly slow in dev mode
>>>>>>> I can assure you that Dev/Prod mode works very very swiftly
>>>>>>> on 8.0.15
>>>>>>> for sizable applications. For example, I have a giant
>>>>>>> application
>>>>>>> (thousands of LOC, 50+ jsf views) that has sub-8ms response
>>>>>>> times
>>>>>>> (minus database time). More than likely they are doing
>>>>>>> something they
>>>>>>> should not be doing with JSF apis and are causing problems.
>>>>>>>
>>>>>>> I would check the obvious things first with VisualVM: heap
>>>>>>> size, gc
>>>>>>> frequency, etc. You can also run their profiler and see if
>>>>>>> you can
>>>>>>> spot anything immediately obvious.
>>>>>>>
>>>>>>> After that, the easiest way to locate the code slowing you
>>>>>>> down is
>>>>>>> bisection. Cut half the code out, run, and continue cutting
>>>>>>> until you
>>>>>>> can locate the problematic code. Then after that, isolate a
>>>>>>> test case
>>>>>>> where flipping a boolean flag triggers the issue and post
>>>>>>> the results
>>>>>>> here.
>>>>>>>
>>>>>>> I would get ahold of: https://www.tomitribe.com who are
>>>>>>> literal
>>>>>>> experts in this stuff. They offer professional services to
>>>>>>> handle
>>>>>>> exactly these situations and can work directly in your
>>>>>>> codebase to
>>>>>>> help you find and fix the problem.
>>>>>>>
>>>>>>> On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski
>>>>>>> <jz...@diegurus.de> wrote:
>>>>>>>> Hi altogether,
>>>>>>>>
>>>>>>>> I'm new to this list and hope it's the right place to ask
>>>>>>>> this
>>>> question.
>>>>>>>> If not, and someone know the right place to ask this,
>>>>>>>> please give
>>>> advise.
>>>>>>>> I'm developing a big JSF Application for a customer. It's
>>>>>>>> a long
>>>> running
>>>>>>>> project and development and when I started with it, TomEE
>>>>>>>> 8.0.6 was
>>>> the
>>>>>>>> most recent Version of TomEE. The switch to Jakarta is
>>>>>>>> planned for
>>>> next
>>>>>>>> year, so I'm still on the 8.x path for the time being and
>>>>>>>> wasn't able
>>>> to
>>>>>>>> test if this problem still exists in the 9.x branch.
>>>>>>>>
>>>>>>>> The problem:
>>>>>>>> On the customers site I'm with the latest TomEE version
>>>>>>>> 8.0.15
>>>> (running
>>>>>>>> on Java 11) and everything works fine, because it is
>>>>>>>> running in
>>>>>>>> production mode. But in my development environment I'm
>>>>>>>> still stuck
>>>> with
>>>>>>>> 8.0.6, because there I need the development mode
>>>>>>>> (javax.faces.PROJECT_STAGE: Development). I need to be
>>>>>>>> able to see
>>>>>>>> changes in facelets without restarting everything
>>>>>>>> everytime, and to
>>>> get
>>>>>>>> more detailed error messages.
>>>>>>>> My attempts to update my dev TomEE to something newer
>>>>>>>> than 8.0.6 all
>>>>>>>> failed, because all (at least all I've tested so far)
>>>>>>>> newer versions
>>>> are
>>>>>>>> so incredibly slow in dev mode, that it's unbearable to
>>>>>>>> use the
>>>>>>>> application. Several very looong seconds on every request
>>>>>>>> (even the
>>>>>>>> little AJAX requests in a view) is simply not a practical
>>>>>>>> environment.
>>>>>>>> After switching to Production mode everything works very
>>>>>>>> fast, and all
>>>>>>>> my views have response times of very few ms (even the big
>>>>>>>> ones with
>>>> max.
>>>>>>>> around 400ms). When switching back to Development mode I
>>>>>>>> have response
>>>>>>>> times of up to 30s on big views. Not with 8.0.6, there
>>>>>>>> even in
>>>>>>>> development mode it's reasonable fast in not getting
>>>>>>>> higher than 1s.
>>>>>>>>
>>>>>>>> My question:
>>>>>>>> What causes this enormous performance degradation? I'm
>>>>>>>> hoping, it is
>>>>>>>> just a configuration which now has another default value
>>>>>>>> or the like.
>>>> If
>>>>>>>> yes, maybe someone can point me in the right direction of
>>>>>>>> where to
>>>> find
>>>>>>>> this configuration? If it's not a configuration thing:
>>>>>>>> what can I do
>>>> to
>>>>>>>> get around this?
>>>>>>>>
>>>>>>>> I haven't tested all Versions of TomEE from 8.0.6 to
>>>>>>>> 8.0.15, so I
>>>> can't
>>>>>>>> say at what version exactly this behaviour changes. If it
>>>>>>>> helps or is
>>>>>>>> needed, I can make some tests to find the exact version
>>>>>>>> where this
>>>>>>>> happens. Versions I've already tested are: 8.0.12, 8.0.13
>>>>>>>> and 8.0.15.
>>>>>>>> All of them are very slow in dev mode. If you need more
>>>>>>>> information,
>>>>>>>> I'll try to provide it.
>>>>>>>>
>>>>>>>> Thanks in advance for any help
>>>>>>>> cu
>>>>>>>> Jens
>>>>>>>>
>>>>>>>>
>>>> --
>>>> jens zurawski
>>>> diegurus - zurawski zurawski poppl rohland GbR
>>>> juister straße 3
>>>> 65199 wiesbaden
>>>>
>>>> kaspersweg 7b
>>>> 26131 oldenburg
>>>>
>>>> internet http://www.diegurus.de
>>>>
>>>> tel +49(0)611 72437966
>>>>
>>>>
>>>> CONFIDENTIALITY NOTICE: This e-mail message is intended only for
>>>> the
>>>> person or entity to which it is addressed and may contain
>>>> confidential
>>>> and/or privileged material. Any unauthorized review, use,
>>>> disclosure or
>>>> distribution is prohibited. If you are not the intended
>>>> recipient,
>>>> please contact the sender by reply e-mail and destroy all copies
>>>> of the
>>>> original message. If you are the intended recipient but do not
>>>> wish to
>>>> receive communications through this medium, please so advise the
>>>> sender
>>>> immediately.
>>>>
>>>>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Jens Zurawski <jz...@diegurus.de>.
Hi Richard,
well, my inner monk would happily walk such routes all day long (and
nights, if it were possible) :-D
Too bad I have to code some "normal applications" for a living, this
limits my monk somehow.
But I really appreciate your kind offer. In fact, it's the first time
I've heard of "git bisect" and after a short reading into this, this is
an interesting method I should definitely add to my tool box.
Furthermore, until now I've only downloaded and used the released
versions of TomEE. To build one by myself would be a nice exercise to
broaden my horizons. Also, my Maven skills are a bit limited (I came to
Maven with Netbeans, and I use it mostly in the IDE which does most of
the magic for me).
If all that doesn't put you off (I might have some stupid questions), I
would be happy to accept your offer.
After all, it's not time critical. I have to collect spare time here and
there. But first of all, I will check all TomEE versions from 8.0.6 up
to 8.0.12 to identify the exact version when the problem starts. When
I've found the version I'll give a short call and then you may give me
some hints on how to make a quick build of TomEE. Then, on a rainy
Sunday, I will step through git bisect to find the problematic commit.
Sounds like fun ;-)
cu
Jens
Am 26.10.2023 um 19:48 schrieb Richard Zowalla:
> Hi,
>
> without knowing code or having a reproducer, it might be not that
> obvious to find a reason / answer for the "academic problem" ;-)
>
> If you really want to boil it down, I recommend the following approach:
>
> (1) From your description it seems, that 8.0.6 is the last known "good"
> version.
>
> (2) You've investigated versions starting from 8.0.12 onwards, which
> seems to be "bad".
>
> Following the approach recommended by Jonathan:
>
> - Can you check if it happens in 8.0.7 (8.0.8, ...) until we find the
> first "bad" released version?
>
> - If we have that, you could start a "git bisect" session between 8.0.6
> and the first version, which is known to be bad (reducing the amount of
> needed bisect cycles dramatically), so we find a commit / reason (if it
> is in TomEE).
>
> - It would involve some quick builds without executing tests (~10-15min
> each) for each bisect step and testing on your side.
>
> Don't know if your inner monk is willing to go that route, but I am
> happy to help with the related Maven commands for quick building :)
>
> Gruß
> Richard
>
>
>
>
>
> Am Donnerstag, dem 26.10.2023 um 18:00 +0200 schrieb Jens Zurawski:
>> Hi Vicente,
>>
>> thank you for the tip. This is a good workaround. When I switch to
>> Production mode and insert this configuration, it reloads the view
>> from
>> the file if it has changed. This means, I can use a decent version of
>> TomEE in my dev environment.
>>
>> However, it doesn't completely solve my problem, as the error
>> reporting
>> isn't as verbose as in Dev mode as far as I can see. But I have to
>> check, maybe it's enough for my development.
>>
>> And, well.... the academic problem: "Why is it still broken in Dev
>> mode?" still runs circles through my brain ;-) Because if it's a
>> problem
>> of my coding, it might be a good idea to change this, before it might
>> get a problem on Production mode, too, at some time in the future.
>>
>> But anyway, you've helped me a good step forward, thanks.
>> cu
>> Jens
>>
>>
>> Am 26.10.2023 um 17:04 schrieb Vicente Rossello:
>>> We are using the production stage with
>>> javax.faces.FACELETS_REFRESH_PERIOD
>>> = 1. It's faster and more similar to production, just try it if it
>>> solves
>>> your problem
>>>
>>> On Thu, Oct 26, 2023 at 4:18 PM Jens Zurawski<jz...@diegurus.de>
>>> wrote:
>>>
>>>> The one bundled with TomEE.
>>>> myfaces 2.3.7 on TomEE 8.0.6
>>>> and e.g. myfaces 2.3.9 on TomEE 8.0.12
>>>>
>>>> Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
>>>>> Which JSF lib are you using?
>>>>>
>>>>> Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens Zurawski
>>>>> <jzu@diegurus.de
>>>>> :
>>>>>> Thank you for your quick reply, Jonathan.
>>>>>>
>>>>>> It is good to know, that normally the Dev mode should work as
>>>>>> fast as
>>>> before even with 8.0.15. So it obviously has something to do with
>>>> my
>>>> environment, configuration and/or code.
>>>>>> And also thank you for the reference to tomitribe.
>>>>>> Unfortunately I'm
>>>> the only developer on this project, and it will be difficult to
>>>> convince my
>>>> customer to an additional support subscription, because he on
>>>> production
>>>> doesn't have issues ;-) But, ok, that's my problem, and maybe
>>>> I'll consult
>>>> tomitribe in the future for other projects.
>>>>>> So, I'm afraid, for the near future I'm on my own to solve
>>>>>> this
>>>> problem. Therefore any hints to restrict the places where I
>>>> should take a
>>>> look will be more than welcome.
>>>>>> I'm a senior programmer with 20 years of experience in Java
>>>>>> programming
>>>> (SE and JSP). But I'm relative new to JSF/JavaEE/JakartaEE and
>>>> sometimes I
>>>> have a hard time to understand which component of the whole stack
>>>> is
>>>> responsible for what.
>>>>>> The project meanwhile has over a 1/4 million lines of code
>>>>>> (most of
>>>> them in the back-end, but also the front-end is big enough to not
>>>> trace
>>>> through every lane if I want to get to the point in this life :-D
>>>> ).
>>>>>> The application now runs for nearly 2 years on the customers
>>>>>> site and
>>>> is very reliable and fast in its daily usage. Apart from
>>>> occasional updates
>>>> every few month, it will never be restarted or need any manual
>>>> maintenance
>>>> at all. So I assume the code is not the badest on earth. At least
>>>> because
>>>> of that I'm able to out-rule things like memory leaks, gc issues
>>>> or
>>>> bottlenecks (max Heap is never reached, neither half of it, even
>>>> after
>>>> running for month).
>>>>>> Of course there could be aspects of JSF which I may not have
>>>>>> understood
>>>> correctly until now. But because there are so many new things to
>>>> a JSF
>>>> beginner like me, I'm running out of ideas what this could be. It
>>>> is
>>>> something that worked well 'til TomEE 8.0.6 (Prod and Dev mode)
>>>> and
>>>> suddenly don't work anymore on 8.0.12 in Dev mode (but still
>>>> works great in
>>>> Prod mode). There are only minor versions in-between. So I think
>>>> there were
>>>> no revolutionary changes anywhere. My hope was (and still is),
>>>> that anybody
>>>> here on the list might have had a similar problem and has solved
>>>> it, or
>>>> maybe someone knows about some changes in Dev mode which could
>>>> explain such
>>>> behaviour (don't know, maybe xml parsing or component tree
>>>> generation has
>>>> changed, or scope handling of managed beans, or whatever could
>>>> cause some
>>>> code to be entered far more often than before, or some cache/pool
>>>> was
>>>> removed or has different behaviour... such things). Then it would
>>>> be far
>>>> more easy for me to track down the real problem.
>>>>>> Anyway, when I get some spare time (currently have some
>>>>>> deadlines in
>>>> sight), I'll try to build a small test case. Either I can get a
>>>> grip on the
>>>> issue myself on doing this, or I can supply it to anyone who
>>>> might want to
>>>> have a look at it.
>>>>>> cu
>>>>>> Jens
>>>>>>
>>>>>>
>>>>>>
>>>>>> Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
>>>>>>>> newer versions are so incredibly slow in dev mode
>>>>>>> I can assure you that Dev/Prod mode works very very swiftly
>>>>>>> on 8.0.15
>>>>>>> for sizable applications. For example, I have a giant
>>>>>>> application
>>>>>>> (thousands of LOC, 50+ jsf views) that has sub-8ms response
>>>>>>> times
>>>>>>> (minus database time). More than likely they are doing
>>>>>>> something they
>>>>>>> should not be doing with JSF apis and are causing problems.
>>>>>>>
>>>>>>> I would check the obvious things first with VisualVM: heap
>>>>>>> size, gc
>>>>>>> frequency, etc. You can also run their profiler and see if
>>>>>>> you can
>>>>>>> spot anything immediately obvious.
>>>>>>>
>>>>>>> After that, the easiest way to locate the code slowing you
>>>>>>> down is
>>>>>>> bisection. Cut half the code out, run, and continue cutting
>>>>>>> until you
>>>>>>> can locate the problematic code. Then after that, isolate a
>>>>>>> test case
>>>>>>> where flipping a boolean flag triggers the issue and post
>>>>>>> the results
>>>>>>> here.
>>>>>>>
>>>>>>> I would get ahold of:https://www.tomitribe.com who are
>>>>>>> literal
>>>>>>> experts in this stuff. They offer professional services to
>>>>>>> handle
>>>>>>> exactly these situations and can work directly in your
>>>>>>> codebase to
>>>>>>> help you find and fix the problem.
>>>>>>>
>>>>>>> On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski
>>>>>>> <jz...@diegurus.de> wrote:
>>>>>>>> Hi altogether,
>>>>>>>>
>>>>>>>> I'm new to this list and hope it's the right place to ask
>>>>>>>> this
>>>> question.
>>>>>>>> If not, and someone know the right place to ask this,
>>>>>>>> please give
>>>> advise.
>>>>>>>> I'm developing a big JSF Application for a customer. It's
>>>>>>>> a long
>>>> running
>>>>>>>> project and development and when I started with it, TomEE
>>>>>>>> 8.0.6 was
>>>> the
>>>>>>>> most recent Version of TomEE. The switch to Jakarta is
>>>>>>>> planned for
>>>> next
>>>>>>>> year, so I'm still on the 8.x path for the time being and
>>>>>>>> wasn't able
>>>> to
>>>>>>>> test if this problem still exists in the 9.x branch.
>>>>>>>>
>>>>>>>> The problem:
>>>>>>>> On the customers site I'm with the latest TomEE version
>>>>>>>> 8.0.15
>>>> (running
>>>>>>>> on Java 11) and everything works fine, because it is
>>>>>>>> running in
>>>>>>>> production mode. But in my development environment I'm
>>>>>>>> still stuck
>>>> with
>>>>>>>> 8.0.6, because there I need the development mode
>>>>>>>> (javax.faces.PROJECT_STAGE: Development). I need to be
>>>>>>>> able to see
>>>>>>>> changes in facelets without restarting everything
>>>>>>>> everytime, and to
>>>> get
>>>>>>>> more detailed error messages.
>>>>>>>> My attempts to update my dev TomEE to something newer
>>>>>>>> than 8.0.6 all
>>>>>>>> failed, because all (at least all I've tested so far)
>>>>>>>> newer versions
>>>> are
>>>>>>>> so incredibly slow in dev mode, that it's unbearable to
>>>>>>>> use the
>>>>>>>> application. Several very looong seconds on every request
>>>>>>>> (even the
>>>>>>>> little AJAX requests in a view) is simply not a practical
>>>>>>>> environment.
>>>>>>>> After switching to Production mode everything works very
>>>>>>>> fast, and all
>>>>>>>> my views have response times of very few ms (even the big
>>>>>>>> ones with
>>>> max.
>>>>>>>> around 400ms). When switching back to Development mode I
>>>>>>>> have response
>>>>>>>> times of up to 30s on big views. Not with 8.0.6, there
>>>>>>>> even in
>>>>>>>> development mode it's reasonable fast in not getting
>>>>>>>> higher than 1s.
>>>>>>>>
>>>>>>>> My question:
>>>>>>>> What causes this enormous performance degradation? I'm
>>>>>>>> hoping, it is
>>>>>>>> just a configuration which now has another default value
>>>>>>>> or the like.
>>>> If
>>>>>>>> yes, maybe someone can point me in the right direction of
>>>>>>>> where to
>>>> find
>>>>>>>> this configuration? If it's not a configuration thing:
>>>>>>>> what can I do
>>>> to
>>>>>>>> get around this?
>>>>>>>>
>>>>>>>> I haven't tested all Versions of TomEE from 8.0.6 to
>>>>>>>> 8.0.15, so I
>>>> can't
>>>>>>>> say at what version exactly this behaviour changes. If it
>>>>>>>> helps or is
>>>>>>>> needed, I can make some tests to find the exact version
>>>>>>>> where this
>>>>>>>> happens. Versions I've already tested are: 8.0.12, 8.0.13
>>>>>>>> and 8.0.15.
>>>>>>>> All of them are very slow in dev mode. If you need more
>>>>>>>> information,
>>>>>>>> I'll try to provide it.
>>>>>>>>
>>>>>>>> Thanks in advance for any help
>>>>>>>> cu
>>>>>>>> Jens
>>>>>>>>
>>>>>>>>
>>>> --
>>>> jens zurawski
>>>> diegurus - zurawski zurawski poppl rohland GbR
>>>> juister straße 3
>>>> 65199 wiesbaden
>>>>
>>>> kaspersweg 7b
>>>> 26131 oldenburg
>>>>
>>>> internethttp://www.diegurus.de
>>>>
>>>> tel +49(0)611 72437966
>>>>
>>>>
>>>> CONFIDENTIALITY NOTICE: This e-mail message is intended only for
>>>> the
>>>> person or entity to which it is addressed and may contain
>>>> confidential
>>>> and/or privileged material. Any unauthorized review, use,
>>>> disclosure or
>>>> distribution is prohibited. If you are not the intended
>>>> recipient,
>>>> please contact the sender by reply e-mail and destroy all copies
>>>> of the
>>>> original message. If you are the intended recipient but do not
>>>> wish to
>>>> receive communications through this medium, please so advise the
>>>> sender
>>>> immediately.
>>>>
>>>>
--
jens zurawski
diegurus - zurawski zurawski poppl rohland GbR
juister straße 3
65199 wiesbaden
kaspersweg 7b
26131 oldenburg
internethttp://www.diegurus.de
tel +49(0)611 72437966
CONFIDENTIALITY NOTICE: This e-mail message is intended only for the
person or entity to which it is addressed and may contain confidential
and/or privileged material. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient,
please contact the sender by reply e-mail and destroy all copies of the
original message. If you are the intended recipient but do not wish to
receive communications through this medium, please so advise the sender
immediately.
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Richard Zowalla <rz...@apache.org>.
Hi,
without knowing code or having a reproducer, it might be not that
obvious to find a reason / answer for the "academic problem" ;-)
If you really want to boil it down, I recommend the following approach:
(1) From your description it seems, that 8.0.6 is the last known "good"
version.
(2) You've investigated versions starting from 8.0.12 onwards, which
seems to be "bad".
Following the approach recommended by Jonathan:
- Can you check if it happens in 8.0.7 (8.0.8, ...) until we find the
first "bad" released version?
- If we have that, you could start a "git bisect" session between 8.0.6
and the first version, which is known to be bad (reducing the amount of
needed bisect cycles dramatically), so we find a commit / reason (if it
is in TomEE).
- It would involve some quick builds without executing tests (~10-15min
each) for each bisect step and testing on your side.
Don't know if your inner monk is willing to go that route, but I am
happy to help with the related Maven commands for quick building :)
Gruß
Richard
Am Donnerstag, dem 26.10.2023 um 18:00 +0200 schrieb Jens Zurawski:
> Hi Vicente,
>
> thank you for the tip. This is a good workaround. When I switch to
> Production mode and insert this configuration, it reloads the view
> from
> the file if it has changed. This means, I can use a decent version of
> TomEE in my dev environment.
>
> However, it doesn't completely solve my problem, as the error
> reporting
> isn't as verbose as in Dev mode as far as I can see. But I have to
> check, maybe it's enough for my development.
>
> And, well.... the academic problem: "Why is it still broken in Dev
> mode?" still runs circles through my brain ;-) Because if it's a
> problem
> of my coding, it might be a good idea to change this, before it might
> get a problem on Production mode, too, at some time in the future.
>
> But anyway, you've helped me a good step forward, thanks.
> cu
> Jens
>
>
> Am 26.10.2023 um 17:04 schrieb Vicente Rossello:
> > We are using the production stage with
> > javax.faces.FACELETS_REFRESH_PERIOD
> > = 1. It's faster and more similar to production, just try it if it
> > solves
> > your problem
> >
> > On Thu, Oct 26, 2023 at 4:18 PM Jens Zurawski <jz...@diegurus.de>
> > wrote:
> >
> > > The one bundled with TomEE.
> > > myfaces 2.3.7 on TomEE 8.0.6
> > > and e.g. myfaces 2.3.9 on TomEE 8.0.12
> > >
> > > Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
> > > > Which JSF lib are you using?
> > > >
> > > > Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens Zurawski
> > > > <jzu@diegurus.de
> > > > :
> > > > > Thank you for your quick reply, Jonathan.
> > > > >
> > > > > It is good to know, that normally the Dev mode should work as
> > > > > fast as
> > > before even with 8.0.15. So it obviously has something to do with
> > > my
> > > environment, configuration and/or code.
> > > > > And also thank you for the reference to tomitribe.
> > > > > Unfortunately I'm
> > > the only developer on this project, and it will be difficult to
> > > convince my
> > > customer to an additional support subscription, because he on
> > > production
> > > doesn't have issues ;-) But, ok, that's my problem, and maybe
> > > I'll consult
> > > tomitribe in the future for other projects.
> > > > > So, I'm afraid, for the near future I'm on my own to solve
> > > > > this
> > > problem. Therefore any hints to restrict the places where I
> > > should take a
> > > look will be more than welcome.
> > > > > I'm a senior programmer with 20 years of experience in Java
> > > > > programming
> > > (SE and JSP). But I'm relative new to JSF/JavaEE/JakartaEE and
> > > sometimes I
> > > have a hard time to understand which component of the whole stack
> > > is
> > > responsible for what.
> > > > > The project meanwhile has over a 1/4 million lines of code
> > > > > (most of
> > > them in the back-end, but also the front-end is big enough to not
> > > trace
> > > through every lane if I want to get to the point in this life :-D
> > > ).
> > > > > The application now runs for nearly 2 years on the customers
> > > > > site and
> > > is very reliable and fast in its daily usage. Apart from
> > > occasional updates
> > > every few month, it will never be restarted or need any manual
> > > maintenance
> > > at all. So I assume the code is not the badest on earth. At least
> > > because
> > > of that I'm able to out-rule things like memory leaks, gc issues
> > > or
> > > bottlenecks (max Heap is never reached, neither half of it, even
> > > after
> > > running for month).
> > > > > Of course there could be aspects of JSF which I may not have
> > > > > understood
> > > correctly until now. But because there are so many new things to
> > > a JSF
> > > beginner like me, I'm running out of ideas what this could be. It
> > > is
> > > something that worked well 'til TomEE 8.0.6 (Prod and Dev mode)
> > > and
> > > suddenly don't work anymore on 8.0.12 in Dev mode (but still
> > > works great in
> > > Prod mode). There are only minor versions in-between. So I think
> > > there were
> > > no revolutionary changes anywhere. My hope was (and still is),
> > > that anybody
> > > here on the list might have had a similar problem and has solved
> > > it, or
> > > maybe someone knows about some changes in Dev mode which could
> > > explain such
> > > behaviour (don't know, maybe xml parsing or component tree
> > > generation has
> > > changed, or scope handling of managed beans, or whatever could
> > > cause some
> > > code to be entered far more often than before, or some cache/pool
> > > was
> > > removed or has different behaviour... such things). Then it would
> > > be far
> > > more easy for me to track down the real problem.
> > > > > Anyway, when I get some spare time (currently have some
> > > > > deadlines in
> > > sight), I'll try to build a small test case. Either I can get a
> > > grip on the
> > > issue myself on doing this, or I can supply it to anyone who
> > > might want to
> > > have a look at it.
> > > > > cu
> > > > > Jens
> > > > >
> > > > >
> > > > >
> > > > > Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
> > > > > > > newer versions are so incredibly slow in dev mode
> > > > > > I can assure you that Dev/Prod mode works very very swiftly
> > > > > > on 8.0.15
> > > > > > for sizable applications. For example, I have a giant
> > > > > > application
> > > > > > (thousands of LOC, 50+ jsf views) that has sub-8ms response
> > > > > > times
> > > > > > (minus database time). More than likely they are doing
> > > > > > something they
> > > > > > should not be doing with JSF apis and are causing problems.
> > > > > >
> > > > > > I would check the obvious things first with VisualVM: heap
> > > > > > size, gc
> > > > > > frequency, etc. You can also run their profiler and see if
> > > > > > you can
> > > > > > spot anything immediately obvious.
> > > > > >
> > > > > > After that, the easiest way to locate the code slowing you
> > > > > > down is
> > > > > > bisection. Cut half the code out, run, and continue cutting
> > > > > > until you
> > > > > > can locate the problematic code. Then after that, isolate a
> > > > > > test case
> > > > > > where flipping a boolean flag triggers the issue and post
> > > > > > the results
> > > > > > here.
> > > > > >
> > > > > > I would get ahold of: https://www.tomitribe.com who are
> > > > > > literal
> > > > > > experts in this stuff. They offer professional services to
> > > > > > handle
> > > > > > exactly these situations and can work directly in your
> > > > > > codebase to
> > > > > > help you find and fix the problem.
> > > > > >
> > > > > > On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski
> > > > > > <jz...@diegurus.de> wrote:
> > > > > > > Hi altogether,
> > > > > > >
> > > > > > > I'm new to this list and hope it's the right place to ask
> > > > > > > this
> > > question.
> > > > > > > If not, and someone know the right place to ask this,
> > > > > > > please give
> > > advise.
> > > > > > > I'm developing a big JSF Application for a customer. It's
> > > > > > > a long
> > > running
> > > > > > > project and development and when I started with it, TomEE
> > > > > > > 8.0.6 was
> > > the
> > > > > > > most recent Version of TomEE. The switch to Jakarta is
> > > > > > > planned for
> > > next
> > > > > > > year, so I'm still on the 8.x path for the time being and
> > > > > > > wasn't able
> > > to
> > > > > > > test if this problem still exists in the 9.x branch.
> > > > > > >
> > > > > > > The problem:
> > > > > > > On the customers site I'm with the latest TomEE version
> > > > > > > 8.0.15
> > > (running
> > > > > > > on Java 11) and everything works fine, because it is
> > > > > > > running in
> > > > > > > production mode. But in my development environment I'm
> > > > > > > still stuck
> > > with
> > > > > > > 8.0.6, because there I need the development mode
> > > > > > > (javax.faces.PROJECT_STAGE: Development). I need to be
> > > > > > > able to see
> > > > > > > changes in facelets without restarting everything
> > > > > > > everytime, and to
> > > get
> > > > > > > more detailed error messages.
> > > > > > > My attempts to update my dev TomEE to something newer
> > > > > > > than 8.0.6 all
> > > > > > > failed, because all (at least all I've tested so far)
> > > > > > > newer versions
> > > are
> > > > > > > so incredibly slow in dev mode, that it's unbearable to
> > > > > > > use the
> > > > > > > application. Several very looong seconds on every request
> > > > > > > (even the
> > > > > > > little AJAX requests in a view) is simply not a practical
> > > > > > > environment.
> > > > > > > After switching to Production mode everything works very
> > > > > > > fast, and all
> > > > > > > my views have response times of very few ms (even the big
> > > > > > > ones with
> > > max.
> > > > > > > around 400ms). When switching back to Development mode I
> > > > > > > have response
> > > > > > > times of up to 30s on big views. Not with 8.0.6, there
> > > > > > > even in
> > > > > > > development mode it's reasonable fast in not getting
> > > > > > > higher than 1s.
> > > > > > >
> > > > > > > My question:
> > > > > > > What causes this enormous performance degradation? I'm
> > > > > > > hoping, it is
> > > > > > > just a configuration which now has another default value
> > > > > > > or the like.
> > > If
> > > > > > > yes, maybe someone can point me in the right direction of
> > > > > > > where to
> > > find
> > > > > > > this configuration? If it's not a configuration thing:
> > > > > > > what can I do
> > > to
> > > > > > > get around this?
> > > > > > >
> > > > > > > I haven't tested all Versions of TomEE from 8.0.6 to
> > > > > > > 8.0.15, so I
> > > can't
> > > > > > > say at what version exactly this behaviour changes. If it
> > > > > > > helps or is
> > > > > > > needed, I can make some tests to find the exact version
> > > > > > > where this
> > > > > > > happens. Versions I've already tested are: 8.0.12, 8.0.13
> > > > > > > and 8.0.15.
> > > > > > > All of them are very slow in dev mode. If you need more
> > > > > > > information,
> > > > > > > I'll try to provide it.
> > > > > > >
> > > > > > > Thanks in advance for any help
> > > > > > > cu
> > > > > > > Jens
> > > > > > >
> > > > > > >
> > > --
> > > jens zurawski
> > > diegurus - zurawski zurawski poppl rohland GbR
> > > juister straße 3
> > > 65199 wiesbaden
> > >
> > > kaspersweg 7b
> > > 26131 oldenburg
> > >
> > > internet http://www.diegurus.de
> > >
> > > tel +49(0)611 72437966
> > >
> > >
> > > CONFIDENTIALITY NOTICE: This e-mail message is intended only for
> > > the
> > > person or entity to which it is addressed and may contain
> > > confidential
> > > and/or privileged material. Any unauthorized review, use,
> > > disclosure or
> > > distribution is prohibited. If you are not the intended
> > > recipient,
> > > please contact the sender by reply e-mail and destroy all copies
> > > of the
> > > original message. If you are the intended recipient but do not
> > > wish to
> > > receive communications through this medium, please so advise the
> > > sender
> > > immediately.
> > >
> > >
>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Jens Zurawski <jz...@diegurus.de>.
Hi Vicente,
thank you for the tip. This is a good workaround. When I switch to
Production mode and insert this configuration, it reloads the view from
the file if it has changed. This means, I can use a decent version of
TomEE in my dev environment.
However, it doesn't completely solve my problem, as the error reporting
isn't as verbose as in Dev mode as far as I can see. But I have to
check, maybe it's enough for my development.
And, well.... the academic problem: "Why is it still broken in Dev
mode?" still runs circles through my brain ;-) Because if it's a problem
of my coding, it might be a good idea to change this, before it might
get a problem on Production mode, too, at some time in the future.
But anyway, you've helped me a good step forward, thanks.
cu
Jens
Am 26.10.2023 um 17:04 schrieb Vicente Rossello:
> We are using the production stage with javax.faces.FACELETS_REFRESH_PERIOD
> = 1. It's faster and more similar to production, just try it if it solves
> your problem
>
> On Thu, Oct 26, 2023 at 4:18 PM Jens Zurawski <jz...@diegurus.de> wrote:
>
>> The one bundled with TomEE.
>> myfaces 2.3.7 on TomEE 8.0.6
>> and e.g. myfaces 2.3.9 on TomEE 8.0.12
>>
>> Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
>>> Which JSF lib are you using?
>>>
>>> Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens Zurawski <jzu@diegurus.de
>>> :
>>>> Thank you for your quick reply, Jonathan.
>>>>
>>>> It is good to know, that normally the Dev mode should work as fast as
>> before even with 8.0.15. So it obviously has something to do with my
>> environment, configuration and/or code.
>>>> And also thank you for the reference to tomitribe. Unfortunately I'm
>> the only developer on this project, and it will be difficult to convince my
>> customer to an additional support subscription, because he on production
>> doesn't have issues ;-) But, ok, that's my problem, and maybe I'll consult
>> tomitribe in the future for other projects.
>>>> So, I'm afraid, for the near future I'm on my own to solve this
>> problem. Therefore any hints to restrict the places where I should take a
>> look will be more than welcome.
>>>> I'm a senior programmer with 20 years of experience in Java programming
>> (SE and JSP). But I'm relative new to JSF/JavaEE/JakartaEE and sometimes I
>> have a hard time to understand which component of the whole stack is
>> responsible for what.
>>>> The project meanwhile has over a 1/4 million lines of code (most of
>> them in the back-end, but also the front-end is big enough to not trace
>> through every lane if I want to get to the point in this life :-D ).
>>>> The application now runs for nearly 2 years on the customers site and
>> is very reliable and fast in its daily usage. Apart from occasional updates
>> every few month, it will never be restarted or need any manual maintenance
>> at all. So I assume the code is not the badest on earth. At least because
>> of that I'm able to out-rule things like memory leaks, gc issues or
>> bottlenecks (max Heap is never reached, neither half of it, even after
>> running for month).
>>>> Of course there could be aspects of JSF which I may not have understood
>> correctly until now. But because there are so many new things to a JSF
>> beginner like me, I'm running out of ideas what this could be. It is
>> something that worked well 'til TomEE 8.0.6 (Prod and Dev mode) and
>> suddenly don't work anymore on 8.0.12 in Dev mode (but still works great in
>> Prod mode). There are only minor versions in-between. So I think there were
>> no revolutionary changes anywhere. My hope was (and still is), that anybody
>> here on the list might have had a similar problem and has solved it, or
>> maybe someone knows about some changes in Dev mode which could explain such
>> behaviour (don't know, maybe xml parsing or component tree generation has
>> changed, or scope handling of managed beans, or whatever could cause some
>> code to be entered far more often than before, or some cache/pool was
>> removed or has different behaviour... such things). Then it would be far
>> more easy for me to track down the real problem.
>>>> Anyway, when I get some spare time (currently have some deadlines in
>> sight), I'll try to build a small test case. Either I can get a grip on the
>> issue myself on doing this, or I can supply it to anyone who might want to
>> have a look at it.
>>>> cu
>>>> Jens
>>>>
>>>>
>>>>
>>>> Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
>>>>>> newer versions are so incredibly slow in dev mode
>>>>> I can assure you that Dev/Prod mode works very very swiftly on 8.0.15
>>>>> for sizable applications. For example, I have a giant application
>>>>> (thousands of LOC, 50+ jsf views) that has sub-8ms response times
>>>>> (minus database time). More than likely they are doing something they
>>>>> should not be doing with JSF apis and are causing problems.
>>>>>
>>>>> I would check the obvious things first with VisualVM: heap size, gc
>>>>> frequency, etc. You can also run their profiler and see if you can
>>>>> spot anything immediately obvious.
>>>>>
>>>>> After that, the easiest way to locate the code slowing you down is
>>>>> bisection. Cut half the code out, run, and continue cutting until you
>>>>> can locate the problematic code. Then after that, isolate a test case
>>>>> where flipping a boolean flag triggers the issue and post the results
>>>>> here.
>>>>>
>>>>> I would get ahold of: https://www.tomitribe.com who are literal
>>>>> experts in this stuff. They offer professional services to handle
>>>>> exactly these situations and can work directly in your codebase to
>>>>> help you find and fix the problem.
>>>>>
>>>>> On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski <jz...@diegurus.de> wrote:
>>>>>> Hi altogether,
>>>>>>
>>>>>> I'm new to this list and hope it's the right place to ask this
>> question.
>>>>>> If not, and someone know the right place to ask this, please give
>> advise.
>>>>>> I'm developing a big JSF Application for a customer. It's a long
>> running
>>>>>> project and development and when I started with it, TomEE 8.0.6 was
>> the
>>>>>> most recent Version of TomEE. The switch to Jakarta is planned for
>> next
>>>>>> year, so I'm still on the 8.x path for the time being and wasn't able
>> to
>>>>>> test if this problem still exists in the 9.x branch.
>>>>>>
>>>>>> The problem:
>>>>>> On the customers site I'm with the latest TomEE version 8.0.15
>> (running
>>>>>> on Java 11) and everything works fine, because it is running in
>>>>>> production mode. But in my development environment I'm still stuck
>> with
>>>>>> 8.0.6, because there I need the development mode
>>>>>> (javax.faces.PROJECT_STAGE: Development). I need to be able to see
>>>>>> changes in facelets without restarting everything everytime, and to
>> get
>>>>>> more detailed error messages.
>>>>>> My attempts to update my dev TomEE to something newer than 8.0.6 all
>>>>>> failed, because all (at least all I've tested so far) newer versions
>> are
>>>>>> so incredibly slow in dev mode, that it's unbearable to use the
>>>>>> application. Several very looong seconds on every request (even the
>>>>>> little AJAX requests in a view) is simply not a practical environment.
>>>>>> After switching to Production mode everything works very fast, and all
>>>>>> my views have response times of very few ms (even the big ones with
>> max.
>>>>>> around 400ms). When switching back to Development mode I have response
>>>>>> times of up to 30s on big views. Not with 8.0.6, there even in
>>>>>> development mode it's reasonable fast in not getting higher than 1s.
>>>>>>
>>>>>> My question:
>>>>>> What causes this enormous performance degradation? I'm hoping, it is
>>>>>> just a configuration which now has another default value or the like.
>> If
>>>>>> yes, maybe someone can point me in the right direction of where to
>> find
>>>>>> this configuration? If it's not a configuration thing: what can I do
>> to
>>>>>> get around this?
>>>>>>
>>>>>> I haven't tested all Versions of TomEE from 8.0.6 to 8.0.15, so I
>> can't
>>>>>> say at what version exactly this behaviour changes. If it helps or is
>>>>>> needed, I can make some tests to find the exact version where this
>>>>>> happens. Versions I've already tested are: 8.0.12, 8.0.13 and 8.0.15.
>>>>>> All of them are very slow in dev mode. If you need more information,
>>>>>> I'll try to provide it.
>>>>>>
>>>>>> Thanks in advance for any help
>>>>>> cu
>>>>>> Jens
>>>>>>
>>>>>>
>> --
>> jens zurawski
>> diegurus - zurawski zurawski poppl rohland GbR
>> juister straße 3
>> 65199 wiesbaden
>>
>> kaspersweg 7b
>> 26131 oldenburg
>>
>> internet http://www.diegurus.de
>>
>> tel +49(0)611 72437966
>>
>>
>> CONFIDENTIALITY NOTICE: This e-mail message is intended only for the
>> person or entity to which it is addressed and may contain confidential
>> and/or privileged material. Any unauthorized review, use, disclosure or
>> distribution is prohibited. If you are not the intended recipient,
>> please contact the sender by reply e-mail and destroy all copies of the
>> original message. If you are the intended recipient but do not wish to
>> receive communications through this medium, please so advise the sender
>> immediately.
>>
>>
--
jens zurawski
diegurus - zurawski zurawski poppl rohland GbR
juister straße 3
65199 wiesbaden
kaspersweg 7b
26131 oldenburg
internet http://www.diegurus.de
tel +49(0)611 72437966
CONFIDENTIALITY NOTICE: This e-mail message is intended only for the
person or entity to which it is addressed and may contain confidential
and/or privileged material. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient,
please contact the sender by reply e-mail and destroy all copies of the
original message. If you are the intended recipient but do not wish to
receive communications through this medium, please so advise the sender
immediately.
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Vicente Rossello <co...@gmail.com>.
We are using the production stage with javax.faces.FACELETS_REFRESH_PERIOD
= 1. It's faster and more similar to production, just try it if it solves
your problem
On Thu, Oct 26, 2023 at 4:18 PM Jens Zurawski <jz...@diegurus.de> wrote:
> The one bundled with TomEE.
> myfaces 2.3.7 on TomEE 8.0.6
> and e.g. myfaces 2.3.9 on TomEE 8.0.12
>
> Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
> > Which JSF lib are you using?
> >
> > Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens Zurawski <jzu@diegurus.de
> >:
> >> Thank you for your quick reply, Jonathan.
> >>
> >> It is good to know, that normally the Dev mode should work as fast as
> before even with 8.0.15. So it obviously has something to do with my
> environment, configuration and/or code.
> >> And also thank you for the reference to tomitribe. Unfortunately I'm
> the only developer on this project, and it will be difficult to convince my
> customer to an additional support subscription, because he on production
> doesn't have issues ;-) But, ok, that's my problem, and maybe I'll consult
> tomitribe in the future for other projects.
> >> So, I'm afraid, for the near future I'm on my own to solve this
> problem. Therefore any hints to restrict the places where I should take a
> look will be more than welcome.
> >>
> >> I'm a senior programmer with 20 years of experience in Java programming
> (SE and JSP). But I'm relative new to JSF/JavaEE/JakartaEE and sometimes I
> have a hard time to understand which component of the whole stack is
> responsible for what.
> >> The project meanwhile has over a 1/4 million lines of code (most of
> them in the back-end, but also the front-end is big enough to not trace
> through every lane if I want to get to the point in this life :-D ).
> >>
> >> The application now runs for nearly 2 years on the customers site and
> is very reliable and fast in its daily usage. Apart from occasional updates
> every few month, it will never be restarted or need any manual maintenance
> at all. So I assume the code is not the badest on earth. At least because
> of that I'm able to out-rule things like memory leaks, gc issues or
> bottlenecks (max Heap is never reached, neither half of it, even after
> running for month).
> >>
> >> Of course there could be aspects of JSF which I may not have understood
> correctly until now. But because there are so many new things to a JSF
> beginner like me, I'm running out of ideas what this could be. It is
> something that worked well 'til TomEE 8.0.6 (Prod and Dev mode) and
> suddenly don't work anymore on 8.0.12 in Dev mode (but still works great in
> Prod mode). There are only minor versions in-between. So I think there were
> no revolutionary changes anywhere. My hope was (and still is), that anybody
> here on the list might have had a similar problem and has solved it, or
> maybe someone knows about some changes in Dev mode which could explain such
> behaviour (don't know, maybe xml parsing or component tree generation has
> changed, or scope handling of managed beans, or whatever could cause some
> code to be entered far more often than before, or some cache/pool was
> removed or has different behaviour... such things). Then it would be far
> more easy for me to track down the real problem.
> >>
> >> Anyway, when I get some spare time (currently have some deadlines in
> sight), I'll try to build a small test case. Either I can get a grip on the
> issue myself on doing this, or I can supply it to anyone who might want to
> have a look at it.
> >>
> >> cu
> >> Jens
> >>
> >>
> >>
> >> Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
> >>>> newer versions are so incredibly slow in dev mode
> >>> I can assure you that Dev/Prod mode works very very swiftly on 8.0.15
> >>> for sizable applications. For example, I have a giant application
> >>> (thousands of LOC, 50+ jsf views) that has sub-8ms response times
> >>> (minus database time). More than likely they are doing something they
> >>> should not be doing with JSF apis and are causing problems.
> >>>
> >>> I would check the obvious things first with VisualVM: heap size, gc
> >>> frequency, etc. You can also run their profiler and see if you can
> >>> spot anything immediately obvious.
> >>>
> >>> After that, the easiest way to locate the code slowing you down is
> >>> bisection. Cut half the code out, run, and continue cutting until you
> >>> can locate the problematic code. Then after that, isolate a test case
> >>> where flipping a boolean flag triggers the issue and post the results
> >>> here.
> >>>
> >>> I would get ahold of: https://www.tomitribe.com who are literal
> >>> experts in this stuff. They offer professional services to handle
> >>> exactly these situations and can work directly in your codebase to
> >>> help you find and fix the problem.
> >>>
> >>> On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski <jz...@diegurus.de> wrote:
> >>>> Hi altogether,
> >>>>
> >>>> I'm new to this list and hope it's the right place to ask this
> question.
> >>>> If not, and someone know the right place to ask this, please give
> advise.
> >>>>
> >>>> I'm developing a big JSF Application for a customer. It's a long
> running
> >>>> project and development and when I started with it, TomEE 8.0.6 was
> the
> >>>> most recent Version of TomEE. The switch to Jakarta is planned for
> next
> >>>> year, so I'm still on the 8.x path for the time being and wasn't able
> to
> >>>> test if this problem still exists in the 9.x branch.
> >>>>
> >>>> The problem:
> >>>> On the customers site I'm with the latest TomEE version 8.0.15
> (running
> >>>> on Java 11) and everything works fine, because it is running in
> >>>> production mode. But in my development environment I'm still stuck
> with
> >>>> 8.0.6, because there I need the development mode
> >>>> (javax.faces.PROJECT_STAGE: Development). I need to be able to see
> >>>> changes in facelets without restarting everything everytime, and to
> get
> >>>> more detailed error messages.
> >>>> My attempts to update my dev TomEE to something newer than 8.0.6 all
> >>>> failed, because all (at least all I've tested so far) newer versions
> are
> >>>> so incredibly slow in dev mode, that it's unbearable to use the
> >>>> application. Several very looong seconds on every request (even the
> >>>> little AJAX requests in a view) is simply not a practical environment.
> >>>> After switching to Production mode everything works very fast, and all
> >>>> my views have response times of very few ms (even the big ones with
> max.
> >>>> around 400ms). When switching back to Development mode I have response
> >>>> times of up to 30s on big views. Not with 8.0.6, there even in
> >>>> development mode it's reasonable fast in not getting higher than 1s.
> >>>>
> >>>> My question:
> >>>> What causes this enormous performance degradation? I'm hoping, it is
> >>>> just a configuration which now has another default value or the like.
> If
> >>>> yes, maybe someone can point me in the right direction of where to
> find
> >>>> this configuration? If it's not a configuration thing: what can I do
> to
> >>>> get around this?
> >>>>
> >>>> I haven't tested all Versions of TomEE from 8.0.6 to 8.0.15, so I
> can't
> >>>> say at what version exactly this behaviour changes. If it helps or is
> >>>> needed, I can make some tests to find the exact version where this
> >>>> happens. Versions I've already tested are: 8.0.12, 8.0.13 and 8.0.15.
> >>>> All of them are very slow in dev mode. If you need more information,
> >>>> I'll try to provide it.
> >>>>
> >>>> Thanks in advance for any help
> >>>> cu
> >>>> Jens
> >>>>
> >>>>
>
> --
> jens zurawski
> diegurus - zurawski zurawski poppl rohland GbR
> juister straße 3
> 65199 wiesbaden
>
> kaspersweg 7b
> 26131 oldenburg
>
> internet http://www.diegurus.de
>
> tel +49(0)611 72437966
>
>
> CONFIDENTIALITY NOTICE: This e-mail message is intended only for the
> person or entity to which it is addressed and may contain confidential
> and/or privileged material. Any unauthorized review, use, disclosure or
> distribution is prohibited. If you are not the intended recipient,
> please contact the sender by reply e-mail and destroy all copies of the
> original message. If you are the intended recipient but do not wish to
> receive communications through this medium, please so advise the sender
> immediately.
>
>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Jens Zurawski <jz...@diegurus.de>.
The one bundled with TomEE.
myfaces 2.3.7 on TomEE 8.0.6
and e.g. myfaces 2.3.9 on TomEE 8.0.12
Am 26.10.2023 um 16:08 schrieb Richard Zowalla:
> Which JSF lib are you using?
>
> Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens Zurawski <jz...@diegurus.de>:
>> Thank you for your quick reply, Jonathan.
>>
>> It is good to know, that normally the Dev mode should work as fast as before even with 8.0.15. So it obviously has something to do with my environment, configuration and/or code.
>> And also thank you for the reference to tomitribe. Unfortunately I'm the only developer on this project, and it will be difficult to convince my customer to an additional support subscription, because he on production doesn't have issues ;-) But, ok, that's my problem, and maybe I'll consult tomitribe in the future for other projects.
>> So, I'm afraid, for the near future I'm on my own to solve this problem. Therefore any hints to restrict the places where I should take a look will be more than welcome.
>>
>> I'm a senior programmer with 20 years of experience in Java programming (SE and JSP). But I'm relative new to JSF/JavaEE/JakartaEE and sometimes I have a hard time to understand which component of the whole stack is responsible for what.
>> The project meanwhile has over a 1/4 million lines of code (most of them in the back-end, but also the front-end is big enough to not trace through every lane if I want to get to the point in this life :-D ).
>>
>> The application now runs for nearly 2 years on the customers site and is very reliable and fast in its daily usage. Apart from occasional updates every few month, it will never be restarted or need any manual maintenance at all. So I assume the code is not the badest on earth. At least because of that I'm able to out-rule things like memory leaks, gc issues or bottlenecks (max Heap is never reached, neither half of it, even after running for month).
>>
>> Of course there could be aspects of JSF which I may not have understood correctly until now. But because there are so many new things to a JSF beginner like me, I'm running out of ideas what this could be. It is something that worked well 'til TomEE 8.0.6 (Prod and Dev mode) and suddenly don't work anymore on 8.0.12 in Dev mode (but still works great in Prod mode). There are only minor versions in-between. So I think there were no revolutionary changes anywhere. My hope was (and still is), that anybody here on the list might have had a similar problem and has solved it, or maybe someone knows about some changes in Dev mode which could explain such behaviour (don't know, maybe xml parsing or component tree generation has changed, or scope handling of managed beans, or whatever could cause some code to be entered far more often than before, or some cache/pool was removed or has different behaviour... such things). Then it would be far more easy for me to track down the real problem.
>>
>> Anyway, when I get some spare time (currently have some deadlines in sight), I'll try to build a small test case. Either I can get a grip on the issue myself on doing this, or I can supply it to anyone who might want to have a look at it.
>>
>> cu
>> Jens
>>
>>
>>
>> Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
>>>> newer versions are so incredibly slow in dev mode
>>> I can assure you that Dev/Prod mode works very very swiftly on 8.0.15
>>> for sizable applications. For example, I have a giant application
>>> (thousands of LOC, 50+ jsf views) that has sub-8ms response times
>>> (minus database time). More than likely they are doing something they
>>> should not be doing with JSF apis and are causing problems.
>>>
>>> I would check the obvious things first with VisualVM: heap size, gc
>>> frequency, etc. You can also run their profiler and see if you can
>>> spot anything immediately obvious.
>>>
>>> After that, the easiest way to locate the code slowing you down is
>>> bisection. Cut half the code out, run, and continue cutting until you
>>> can locate the problematic code. Then after that, isolate a test case
>>> where flipping a boolean flag triggers the issue and post the results
>>> here.
>>>
>>> I would get ahold of: https://www.tomitribe.com who are literal
>>> experts in this stuff. They offer professional services to handle
>>> exactly these situations and can work directly in your codebase to
>>> help you find and fix the problem.
>>>
>>> On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski <jz...@diegurus.de> wrote:
>>>> Hi altogether,
>>>>
>>>> I'm new to this list and hope it's the right place to ask this question.
>>>> If not, and someone know the right place to ask this, please give advise.
>>>>
>>>> I'm developing a big JSF Application for a customer. It's a long running
>>>> project and development and when I started with it, TomEE 8.0.6 was the
>>>> most recent Version of TomEE. The switch to Jakarta is planned for next
>>>> year, so I'm still on the 8.x path for the time being and wasn't able to
>>>> test if this problem still exists in the 9.x branch.
>>>>
>>>> The problem:
>>>> On the customers site I'm with the latest TomEE version 8.0.15 (running
>>>> on Java 11) and everything works fine, because it is running in
>>>> production mode. But in my development environment I'm still stuck with
>>>> 8.0.6, because there I need the development mode
>>>> (javax.faces.PROJECT_STAGE: Development). I need to be able to see
>>>> changes in facelets without restarting everything everytime, and to get
>>>> more detailed error messages.
>>>> My attempts to update my dev TomEE to something newer than 8.0.6 all
>>>> failed, because all (at least all I've tested so far) newer versions are
>>>> so incredibly slow in dev mode, that it's unbearable to use the
>>>> application. Several very looong seconds on every request (even the
>>>> little AJAX requests in a view) is simply not a practical environment.
>>>> After switching to Production mode everything works very fast, and all
>>>> my views have response times of very few ms (even the big ones with max.
>>>> around 400ms). When switching back to Development mode I have response
>>>> times of up to 30s on big views. Not with 8.0.6, there even in
>>>> development mode it's reasonable fast in not getting higher than 1s.
>>>>
>>>> My question:
>>>> What causes this enormous performance degradation? I'm hoping, it is
>>>> just a configuration which now has another default value or the like. If
>>>> yes, maybe someone can point me in the right direction of where to find
>>>> this configuration? If it's not a configuration thing: what can I do to
>>>> get around this?
>>>>
>>>> I haven't tested all Versions of TomEE from 8.0.6 to 8.0.15, so I can't
>>>> say at what version exactly this behaviour changes. If it helps or is
>>>> needed, I can make some tests to find the exact version where this
>>>> happens. Versions I've already tested are: 8.0.12, 8.0.13 and 8.0.15.
>>>> All of them are very slow in dev mode. If you need more information,
>>>> I'll try to provide it.
>>>>
>>>> Thanks in advance for any help
>>>> cu
>>>> Jens
>>>>
>>>>
--
jens zurawski
diegurus - zurawski zurawski poppl rohland GbR
juister straße 3
65199 wiesbaden
kaspersweg 7b
26131 oldenburg
internet http://www.diegurus.de
tel +49(0)611 72437966
CONFIDENTIALITY NOTICE: This e-mail message is intended only for the
person or entity to which it is addressed and may contain confidential
and/or privileged material. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient,
please contact the sender by reply e-mail and destroy all copies of the
original message. If you are the intended recipient but do not wish to
receive communications through this medium, please so advise the sender
immediately.
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Richard Zowalla <ri...@zowalla.com>.
Which JSF lib are you using?
Am 26. Oktober 2023 16:00:57 MESZ schrieb Jens Zurawski <jz...@diegurus.de>:
>Thank you for your quick reply, Jonathan.
>
>It is good to know, that normally the Dev mode should work as fast as before even with 8.0.15. So it obviously has something to do with my environment, configuration and/or code.
>And also thank you for the reference to tomitribe. Unfortunately I'm the only developer on this project, and it will be difficult to convince my customer to an additional support subscription, because he on production doesn't have issues ;-) But, ok, that's my problem, and maybe I'll consult tomitribe in the future for other projects.
>So, I'm afraid, for the near future I'm on my own to solve this problem. Therefore any hints to restrict the places where I should take a look will be more than welcome.
>
>I'm a senior programmer with 20 years of experience in Java programming (SE and JSP). But I'm relative new to JSF/JavaEE/JakartaEE and sometimes I have a hard time to understand which component of the whole stack is responsible for what.
>The project meanwhile has over a 1/4 million lines of code (most of them in the back-end, but also the front-end is big enough to not trace through every lane if I want to get to the point in this life :-D ).
>
>The application now runs for nearly 2 years on the customers site and is very reliable and fast in its daily usage. Apart from occasional updates every few month, it will never be restarted or need any manual maintenance at all. So I assume the code is not the badest on earth. At least because of that I'm able to out-rule things like memory leaks, gc issues or bottlenecks (max Heap is never reached, neither half of it, even after running for month).
>
>Of course there could be aspects of JSF which I may not have understood correctly until now. But because there are so many new things to a JSF beginner like me, I'm running out of ideas what this could be. It is something that worked well 'til TomEE 8.0.6 (Prod and Dev mode) and suddenly don't work anymore on 8.0.12 in Dev mode (but still works great in Prod mode). There are only minor versions in-between. So I think there were no revolutionary changes anywhere. My hope was (and still is), that anybody here on the list might have had a similar problem and has solved it, or maybe someone knows about some changes in Dev mode which could explain such behaviour (don't know, maybe xml parsing or component tree generation has changed, or scope handling of managed beans, or whatever could cause some code to be entered far more often than before, or some cache/pool was removed or has different behaviour... such things). Then it would be far more easy for me to track down the real problem.
>
>Anyway, when I get some spare time (currently have some deadlines in sight), I'll try to build a small test case. Either I can get a grip on the issue myself on doing this, or I can supply it to anyone who might want to have a look at it.
>
>cu
>Jens
>
>
>
>Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
>>> newer versions are so incredibly slow in dev mode
>> I can assure you that Dev/Prod mode works very very swiftly on 8.0.15
>> for sizable applications. For example, I have a giant application
>> (thousands of LOC, 50+ jsf views) that has sub-8ms response times
>> (minus database time). More than likely they are doing something they
>> should not be doing with JSF apis and are causing problems.
>>
>> I would check the obvious things first with VisualVM: heap size, gc
>> frequency, etc. You can also run their profiler and see if you can
>> spot anything immediately obvious.
>>
>> After that, the easiest way to locate the code slowing you down is
>> bisection. Cut half the code out, run, and continue cutting until you
>> can locate the problematic code. Then after that, isolate a test case
>> where flipping a boolean flag triggers the issue and post the results
>> here.
>>
>> I would get ahold of: https://www.tomitribe.com who are literal
>> experts in this stuff. They offer professional services to handle
>> exactly these situations and can work directly in your codebase to
>> help you find and fix the problem.
>>
>> On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski <jz...@diegurus.de> wrote:
>>> Hi altogether,
>>>
>>> I'm new to this list and hope it's the right place to ask this question.
>>> If not, and someone know the right place to ask this, please give advise.
>>>
>>> I'm developing a big JSF Application for a customer. It's a long running
>>> project and development and when I started with it, TomEE 8.0.6 was the
>>> most recent Version of TomEE. The switch to Jakarta is planned for next
>>> year, so I'm still on the 8.x path for the time being and wasn't able to
>>> test if this problem still exists in the 9.x branch.
>>>
>>> The problem:
>>> On the customers site I'm with the latest TomEE version 8.0.15 (running
>>> on Java 11) and everything works fine, because it is running in
>>> production mode. But in my development environment I'm still stuck with
>>> 8.0.6, because there I need the development mode
>>> (javax.faces.PROJECT_STAGE: Development). I need to be able to see
>>> changes in facelets without restarting everything everytime, and to get
>>> more detailed error messages.
>>> My attempts to update my dev TomEE to something newer than 8.0.6 all
>>> failed, because all (at least all I've tested so far) newer versions are
>>> so incredibly slow in dev mode, that it's unbearable to use the
>>> application. Several very looong seconds on every request (even the
>>> little AJAX requests in a view) is simply not a practical environment.
>>> After switching to Production mode everything works very fast, and all
>>> my views have response times of very few ms (even the big ones with max.
>>> around 400ms). When switching back to Development mode I have response
>>> times of up to 30s on big views. Not with 8.0.6, there even in
>>> development mode it's reasonable fast in not getting higher than 1s.
>>>
>>> My question:
>>> What causes this enormous performance degradation? I'm hoping, it is
>>> just a configuration which now has another default value or the like. If
>>> yes, maybe someone can point me in the right direction of where to find
>>> this configuration? If it's not a configuration thing: what can I do to
>>> get around this?
>>>
>>> I haven't tested all Versions of TomEE from 8.0.6 to 8.0.15, so I can't
>>> say at what version exactly this behaviour changes. If it helps or is
>>> needed, I can make some tests to find the exact version where this
>>> happens. Versions I've already tested are: 8.0.12, 8.0.13 and 8.0.15.
>>> All of them are very slow in dev mode. If you need more information,
>>> I'll try to provide it.
>>>
>>> Thanks in advance for any help
>>> cu
>>> Jens
>>>
>>>
>>
>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by Jens Zurawski <jz...@diegurus.de>.
Thank you for your quick reply, Jonathan.
It is good to know, that normally the Dev mode should work as fast as
before even with 8.0.15. So it obviously has something to do with my
environment, configuration and/or code.
And also thank you for the reference to tomitribe. Unfortunately I'm the
only developer on this project, and it will be difficult to convince my
customer to an additional support subscription, because he on production
doesn't have issues ;-) But, ok, that's my problem, and maybe I'll
consult tomitribe in the future for other projects.
So, I'm afraid, for the near future I'm on my own to solve this problem.
Therefore any hints to restrict the places where I should take a look
will be more than welcome.
I'm a senior programmer with 20 years of experience in Java programming
(SE and JSP). But I'm relative new to JSF/JavaEE/JakartaEE and sometimes
I have a hard time to understand which component of the whole stack is
responsible for what.
The project meanwhile has over a 1/4 million lines of code (most of them
in the back-end, but also the front-end is big enough to not trace
through every lane if I want to get to the point in this life :-D ).
The application now runs for nearly 2 years on the customers site and is
very reliable and fast in its daily usage. Apart from occasional updates
every few month, it will never be restarted or need any manual
maintenance at all. So I assume the code is not the badest on earth. At
least because of that I'm able to out-rule things like memory leaks, gc
issues or bottlenecks (max Heap is never reached, neither half of it,
even after running for month).
Of course there could be aspects of JSF which I may not have understood
correctly until now. But because there are so many new things to a JSF
beginner like me, I'm running out of ideas what this could be. It is
something that worked well 'til TomEE 8.0.6 (Prod and Dev mode) and
suddenly don't work anymore on 8.0.12 in Dev mode (but still works great
in Prod mode). There are only minor versions in-between. So I think
there were no revolutionary changes anywhere. My hope was (and still
is), that anybody here on the list might have had a similar problem and
has solved it, or maybe someone knows about some changes in Dev mode
which could explain such behaviour (don't know, maybe xml parsing or
component tree generation has changed, or scope handling of managed
beans, or whatever could cause some code to be entered far more often
than before, or some cache/pool was removed or has different
behaviour... such things). Then it would be far more easy for me to
track down the real problem.
Anyway, when I get some spare time (currently have some deadlines in
sight), I'll try to build a small test case. Either I can get a grip on
the issue myself on doing this, or I can supply it to anyone who might
want to have a look at it.
cu
Jens
Am 26.10.2023 um 14:01 schrieb Jonathan S. Fisher:
>> newer versions are so incredibly slow in dev mode
> I can assure you that Dev/Prod mode works very very swiftly on 8.0.15
> for sizable applications. For example, I have a giant application
> (thousands of LOC, 50+ jsf views) that has sub-8ms response times
> (minus database time). More than likely they are doing something they
> should not be doing with JSF apis and are causing problems.
>
> I would check the obvious things first with VisualVM: heap size, gc
> frequency, etc. You can also run their profiler and see if you can
> spot anything immediately obvious.
>
> After that, the easiest way to locate the code slowing you down is
> bisection. Cut half the code out, run, and continue cutting until you
> can locate the problematic code. Then after that, isolate a test case
> where flipping a boolean flag triggers the issue and post the results
> here.
>
> I would get ahold of: https://www.tomitribe.com who are literal
> experts in this stuff. They offer professional services to handle
> exactly these situations and can work directly in your codebase to
> help you find and fix the problem.
>
> On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski <jz...@diegurus.de> wrote:
>> Hi altogether,
>>
>> I'm new to this list and hope it's the right place to ask this question.
>> If not, and someone know the right place to ask this, please give advise.
>>
>> I'm developing a big JSF Application for a customer. It's a long running
>> project and development and when I started with it, TomEE 8.0.6 was the
>> most recent Version of TomEE. The switch to Jakarta is planned for next
>> year, so I'm still on the 8.x path for the time being and wasn't able to
>> test if this problem still exists in the 9.x branch.
>>
>> The problem:
>> On the customers site I'm with the latest TomEE version 8.0.15 (running
>> on Java 11) and everything works fine, because it is running in
>> production mode. But in my development environment I'm still stuck with
>> 8.0.6, because there I need the development mode
>> (javax.faces.PROJECT_STAGE: Development). I need to be able to see
>> changes in facelets without restarting everything everytime, and to get
>> more detailed error messages.
>> My attempts to update my dev TomEE to something newer than 8.0.6 all
>> failed, because all (at least all I've tested so far) newer versions are
>> so incredibly slow in dev mode, that it's unbearable to use the
>> application. Several very looong seconds on every request (even the
>> little AJAX requests in a view) is simply not a practical environment.
>> After switching to Production mode everything works very fast, and all
>> my views have response times of very few ms (even the big ones with max.
>> around 400ms). When switching back to Development mode I have response
>> times of up to 30s on big views. Not with 8.0.6, there even in
>> development mode it's reasonable fast in not getting higher than 1s.
>>
>> My question:
>> What causes this enormous performance degradation? I'm hoping, it is
>> just a configuration which now has another default value or the like. If
>> yes, maybe someone can point me in the right direction of where to find
>> this configuration? If it's not a configuration thing: what can I do to
>> get around this?
>>
>> I haven't tested all Versions of TomEE from 8.0.6 to 8.0.15, so I can't
>> say at what version exactly this behaviour changes. If it helps or is
>> needed, I can make some tests to find the exact version where this
>> happens. Versions I've already tested are: 8.0.12, 8.0.13 and 8.0.15.
>> All of them are very slow in dev mode. If you need more information,
>> I'll try to provide it.
>>
>> Thanks in advance for any help
>> cu
>> Jens
>>
>>
>
Re: Very slow JSF Development Mode on tomee 8.x newer than 8.0.6
Posted by "Jonathan S. Fisher" <ex...@gmail.com>.
> newer versions are so incredibly slow in dev mode
I can assure you that Dev/Prod mode works very very swiftly on 8.0.15
for sizable applications. For example, I have a giant application
(thousands of LOC, 50+ jsf views) that has sub-8ms response times
(minus database time). More than likely they are doing something they
should not be doing with JSF apis and are causing problems.
I would check the obvious things first with VisualVM: heap size, gc
frequency, etc. You can also run their profiler and see if you can
spot anything immediately obvious.
After that, the easiest way to locate the code slowing you down is
bisection. Cut half the code out, run, and continue cutting until you
can locate the problematic code. Then after that, isolate a test case
where flipping a boolean flag triggers the issue and post the results
here.
I would get ahold of: https://www.tomitribe.com who are literal
experts in this stuff. They offer professional services to handle
exactly these situations and can work directly in your codebase to
help you find and fix the problem.
On Thu, Oct 26, 2023 at 5:48 AM Jens Zurawski <jz...@diegurus.de> wrote:
>
> Hi altogether,
>
> I'm new to this list and hope it's the right place to ask this question.
> If not, and someone know the right place to ask this, please give advise.
>
> I'm developing a big JSF Application for a customer. It's a long running
> project and development and when I started with it, TomEE 8.0.6 was the
> most recent Version of TomEE. The switch to Jakarta is planned for next
> year, so I'm still on the 8.x path for the time being and wasn't able to
> test if this problem still exists in the 9.x branch.
>
> The problem:
> On the customers site I'm with the latest TomEE version 8.0.15 (running
> on Java 11) and everything works fine, because it is running in
> production mode. But in my development environment I'm still stuck with
> 8.0.6, because there I need the development mode
> (javax.faces.PROJECT_STAGE: Development). I need to be able to see
> changes in facelets without restarting everything everytime, and to get
> more detailed error messages.
> My attempts to update my dev TomEE to something newer than 8.0.6 all
> failed, because all (at least all I've tested so far) newer versions are
> so incredibly slow in dev mode, that it's unbearable to use the
> application. Several very looong seconds on every request (even the
> little AJAX requests in a view) is simply not a practical environment.
> After switching to Production mode everything works very fast, and all
> my views have response times of very few ms (even the big ones with max.
> around 400ms). When switching back to Development mode I have response
> times of up to 30s on big views. Not with 8.0.6, there even in
> development mode it's reasonable fast in not getting higher than 1s.
>
> My question:
> What causes this enormous performance degradation? I'm hoping, it is
> just a configuration which now has another default value or the like. If
> yes, maybe someone can point me in the right direction of where to find
> this configuration? If it's not a configuration thing: what can I do to
> get around this?
>
> I haven't tested all Versions of TomEE from 8.0.6 to 8.0.15, so I can't
> say at what version exactly this behaviour changes. If it helps or is
> needed, I can make some tests to find the exact version where this
> happens. Versions I've already tested are: 8.0.12, 8.0.13 and 8.0.15.
> All of them are very slow in dev mode. If you need more information,
> I'll try to provide it.
>
> Thanks in advance for any help
> cu
> Jens
>
>
--
Jonathan | exabrial@gmail.com
Pessimists, see a jar as half empty. Optimists, in contrast, see it as
half full.
Engineers, of course, understand the glass is twice as big as it needs to be.