[jira] [Comment Edited] (RAVE-838) Performance problems after model-split merge

["Chris Geer (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/RAVE-838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13489313#comment-13489313 ] 

Chris Geer edited comment on RAVE-838 at 11/2/12 9:30 AM:
----------------------------------------------------------

Using JProfiler, I was able to profile the JPA calls directly for the first time which is helping shed some light on what's going on. Currently, on 0.17, when loading the default canonical home page, WdigetRepository.get is called 5 times, UserRepository.get is called 5 times, PersonRepository.get is called 3 times and four other JPA queries are run. Compare that with 0.16 where you just have the four JPA queries, no direct calls to any repository.get method.

After digging into it a little more I came to the conclusion that it's not the number of calls that is the sole problem, it's the sheer amount of data loaded on each call. For example, on each widget call there has to be queries to load Widgets, WidgetComments, WidgetRatings, WidgetTags and Categories (missed that split). The frustrating thing is that none of those extra sets of data is actually used in any of the calls to load the page (that I saw), so we are loading lots of data (5 times) that is never used on that page. Now, for a lot of them, there isn't data by default but there is still an attempt to load data. There is the same problem when you look at person and user. As it turns out, JPA has a solution to handle scenarios like this, Lazy fetch, which will only load the sub-data on demand which would be great. When I changed the Widget object to lazy fetch all the sub data the time spent in those calls dropped by ~70%. The major problem this introduced however was that the lazy data was never loaded. I couldn't find a way to fix that but maybe someone else with more JPA experience could have a look or explain why that's a really bad idea. 

EDIT: Looking at the H2 performance, it looks like each request to H2 spawns a new thread (or multiple threads). On the load of the canonical home page H2 spanwed (and killed) 16 separate threads on 0.16 and well over 100 separate threads on 0.17. I can't confirm if those are from a pool or what but that is a lot of short lived threads.
                
      was (Author: geerzo):
    Using JProfiler, I was able to profile the JPA calls directly for the first time which is helping shed some light on what's going on. Currently, on 0.17, when loading the default canonical home page, WdigetRepository.get is called 5 times, UserRepository.get is called 5 times, PersonRepository.get is called 3 times and four other JPA queries are run. Compare that with 0.16 where you just have the four JPA queries, no direct calls to any repository.get method.

After digging into it a little more I came to the conclusion that it's not the number of calls that is the sole problem, it's the sheer amount of data loaded on each call. For example, on each widget call there has to be queries to load Widgets, WidgetComments, WidgetRatings, WidgetTags and Categories (missed that split). The frustrating thing is that none of those extra sets of data is actually used in any of the calls to load the page (that I saw), so we are loading lots of data (5 times) that is never used on that page. Now, for a lot of them, there isn't data by default but there is still an attempt to load data. There is the same problem when you look at person and user. As it turns out, JPA has a solution to handle scenarios like this, Lazy fetch, which will only load the sub-data on demand which would be great. When I changed the Widget object to lazy fetch all the sub data the time spent in those calls dropped by ~70%. The major problem this introduced however was that the lazy data was never loaded. I couldn't find a way to fix that but maybe someone else with more JPA experience could have a look or explain why that's a really bad idea. 
                  
> Performance problems after model-split merge
> --------------------------------------------
>
>                 Key: RAVE-838
>                 URL: https://issues.apache.org/jira/browse/RAVE-838
>             Project: Rave
>          Issue Type: Bug
>    Affects Versions: 0.17
>            Reporter: Chris Geer
>            Assignee: Chris Geer
>             Fix For: 0.18
>
>
>  - WidgetServer/Repository.get is being called much more frequently with the RegionWidget/Widget split.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira