You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@roller.apache.org by Allen Gilliland <al...@sun.com> on 2006/11/13 23:10:28 UTC
Business layer cleanup for 3.2
One of the things that I am planning to do for the 3.2 release is do
some audit/cleanup of the current business layer code. There are a
variety of things which could use improving, but the main goal is to fix
our Hibernate configuration so that we are 1) properly using the open
session in view pattern and 2) enabling lazy fetching on all objects and
associations.
Right now our Hibernate config is pretty messy and doesn't take
advantage of many of Hibernate's performance features, so the main
reason to do this work is to improve the performance of the business
layer. The second big reason is just to reduce clutter and simplify the
code as much as possible. There are plenty of places in the code where
we have methods that aren't used at all or methods which are duplicated,
so those would all be cleaned up.
I have most of this work done already (but not checked in) and there
aren't really any surprise changes that I had to make except when it
came to the hierarchical objects. I tried for multiple days to get the
hierarchical objects to work with the updated hibernate config and the
current data model, but I kept running into problems. So to fix the
problem I had to make a small tweak to the way hierarchical objects are
persisted which fixed my issues and I believe drastically simplifies the
problem overall. The basic change is that I have completely removed the
HierarchicalPersistentObject class and Assoc and it's subclasses and
changed the data model so that we have a more normal hierarchical model.
So, for weblog categories I added a simple 'parentid' column to the
weblogcategory table and that allows a category to manage relationships
between it's parent and children directly. Same goes for the FolderData
class, but as it turns out that column already existed in the schema but
wasn't being used. Upgrade path for both of these is fairly simple and
only requires populating these columns with the right value.
I'm not sure if anyone really wants to see more of a proposal for this,
which is why I started with an adhoc description here on the list. As I
said, I am not actually modifying anything from a feature point of view,
only cleaning up what is already there. If anyone wants to see more
about the changes to the hierarchical objects then I can post them on
the wiki or something.
-- Allen
Re: Business layer cleanup for 3.2
Posted by Allen Gilliland <al...@sun.com>.
k ... this work is now in the trunk, so anyone who is interested can
check it out there or review the changes in revision 475368
-- Allen
Allen Gilliland wrote:
>
>
> Anil Gangolli wrote:
>>
>> This actually has to do with the use of Hibernate Contextual Sessions
>> ( the thread-bound session returned by getCurrentSession() ), not
>> Hibernate's behavior when using JDBC transaction semantics.
>>
>> http://www.hibernate.org/hib_docs/v3/reference/en/html/architecture.html#architecture-current-session
>>
>>
>> We might want to consider using the ManagedSessionContext for this and
>> managing the session binding/opening/closing ourselves; that might be
>> cleaner and more localized than trying to manage flush() ourselves.
>>
>> The problem with managing flush() explicitly is that it tends to lead
>> to having to assume boundaries in composition of operations (where the
>> flush() is called). For example, if you query as part of a method you
>> have to flush or know that flush is not necessary.
>
> I am definitely interested in exploring this more. I have completed
> what I originally set out to do and so far from my testing everything
> seems to be working properly and the load testing that I have done has
> shown some improvements here and there, but not all that I was hoping
> for. Part of the reason there wasn't as big of a performance
> improvement definitely may be due to the use of FlushMode.NEVER, so I
> plan to investigate that as best I can and see if there are better
> alternatives.
>
> If you have any other ideas then definitely send them out and I will
> attempt to try them out and see what kind of impact they have.
>
>
>>
>>
>> Also, looking back at the initial message, I was confused by the
>> phrase "2) enabling lazy fetching on all objects and
>> associations" in Allen's message. Did you mean all single and
>> collection-valued associations? I don't think lazy fetching of object
>> properties will be a win at all.
>
> Good point. Yes, I meant only for single and collection-valued
> associations, which is what I believe Hibernate defaults to right now.
> Now that I have everything working I plan to go back and do a bit more
> rigorous debugging to double check this and make sure, but I am pretty
> sure this is the case.
>
> In any case, I would agree that lazy fetching for all properties would
> be silly.
>
>
>>
>> We might also consider using join fetching for certain associations,
>> particularly around weblog entry data. Maybe you looked at this already?
>
> Nope, I haven't gotten that far. I am certainly open to this though, so
> if you have time to offer up a few more details I can try experimenting
> with it.
>
> -- Allen
>
>
>>
>> --a.
>>
>>
>> ----- Original Message ----- From: "Allen Gilliland"
>> <al...@sun.com>
>> To: <ro...@incubator.apache.org>
>> Sent: Tuesday, November 14, 2006 11:16 AM
>> Subject: Re: Business layer cleanup for 3.2
>>
>>
>>>
>>>
>>> Craig L Russell wrote:
>>>>
>>>> On Nov 14, 2006, at 9:48 AM, Allen Gilliland wrote:
>>>>
>>>>>
>>>>>
>>>>>> It's not unlikely that we will break some things temporarily and
>>>>>> not notice it for a while. The riskiest aspect to me in this
>>>>>> regard is lazy fetching because it really demands that the session
>>>>>> span the entire request, which we seemed to have a hard time doing
>>>>>> properly earlier, and I'm not sure exactly why. We backed out of
>>>>>> lazy fetching just before one release a ways back because we would
>>>>>> hit odd session closed exceptions that we didn't have time to
>>>>>> figure out. It's possible that some of Allen's earlier session
>>>>>> management cleanups already got us out of those issues. It's a
>>>>>> good idea to revisit this now. I think that is also likely to
>>>>>> make us more portable to optimizations in other persistence
>>>>>> implementations that expect essentially the same session
>>>>>> management pattern.
>>>>>
>>>>> so far I have been able to make all the changes that I think are
>>>>> correct and I have all the unit tests running correctly, so what I
>>>>> am doing now is going over the actual webapp and running through
>>>>> all the operations that I can to find and fix anything that I
>>>>> find. Things are definitely cropping up, but so far the lazy
>>>>> initialization problem hasn't come up.
>>>>>
>>>>> The bigger problem has been caused by changing the hibernate config
>>>>> to use FlushMode.NEVER, which means that hibernate doesn't flush
>>>>> its state to the db until we explicitly call the flush() method on
>>>>> the Session. As it turns out, a *lot* of the stuff we were doing
>>>>> has been very reliant on auto flushing for it to work, so there
>>>>> have been a handful of places where I have had to figure out how to
>>>>> fix that problem. So far so good though, and I hope to have things
>>>>> cleaned up enough to commit in the next day or so.
>>>>
>>>> What is the reason you want to set hibernate config to use
>>>> FlushMode.NEVER? From what I know of it, this is an antipattern.
>>>
>>> I will do some more reading about it because if it is an antipattern
>>> then I should reconsider it, but the main reason why I am trying this
>>> now is to get around some of Hibernates automation. The main problem
>>> being that Hibernate's default handling for JDBCTransactions is setup
>>> such that whenever you commit a transaction it closes the Session
>>> that was handling that transaction, and that causes a problem with
>>> lazy initialization because objects can no longer access associations
>>> if the Session they were attached to is closed. I sifted through
>>> things on the Hibernate forums to find other folks with the same
>>> problem and how they fixed it and that's where I came up with
>>> FlushMode.NEVER.
>>>
>>> It may be possible that we can accomplish that without using
>>> FlushMode.NEVER, but i'll have to look at it more carefully to see.
>>>
>>> Regardless of whether or not we use it I still think that in
>>> principal it should work. The places where I have found things
>>> broken by this change is situations where we are kind of
>>> circumventing our object model and going directly to the db which
>>> causes a slight disconnect. i.e. if I call saveObject(foo) and then
>>> later in the same transaction try to getObject(foo) via a query
>>> before it has been flushed to the db. So far these scenarios look
>>> more like slightly incorrect code as much as anything else, but I am
>>> still investigating.
>>>
>>> I'll look into this more today though.
>>>
>>> -- Allen
>>>
>>>
>>>>
>>>> Craig
>>>>>
>>>>> -- Allen
>>>>>
>>>>>
>>>>>> --a.
>>>>>> ----- Original Message ----- From: "Allen Gilliland"
>>>>>> <Al...@Sun.COM>
>>>>>> To: <ro...@incubator.apache.org>
>>>>>> Sent: Monday, November 13, 2006 4:46 PM
>>>>>> Subject: Re: Business layer cleanup for 3.2
>>>>>>> certainly. I didn't do much in the way of renaming things yet,
>>>>>>> the first pass was mainly about fixing up the Hibernate config to
>>>>>>> work the way I think it should have been working and clearing out
>>>>>>> some things along the way. Once that is done then I plan to go
>>>>>>> over the business layer more times and find places where methods
>>>>>>> should be renamed, removed, or consolidated in any way. I also
>>>>>>> want to keep building on the unit tests because I think they are
>>>>>>> pretty good now, but there are a few gaps here and there.
>>>>>>>
>>>>>>> At the end of the day this work will definitely help to make the
>>>>>>> work on the JDO/JPA backends quite a bit easier.
>>>>>>>
>>>>>>> -- Allen
>>>>>>>
>>>>>>>
>>>>>>> Craig L Russell wrote:
>>>>>>>> Hi Allen,
>>>>>>>>
>>>>>>>> We had discussed a number of issues with the manager classes
>>>>>>>> such as misspelled method names and incomplete functionality
>>>>>>>> (having the caller iterate through collections).
>>>>>>>>
>>>>>>>> I'd be happy to review what you've done in terms of cleanup.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Craig
>>>>>>>>
>>>>>>>> On Nov 13, 2006, at 2:10 PM, Allen Gilliland wrote:
>>>>>>>>
>>>>>>>>> One of the things that I am planning to do for the 3.2 release
>>>>>>>>> is do some audit/cleanup of the current business layer code.
>>>>>>>>> There are a variety of things which could use improving, but
>>>>>>>>> the main goal is to fix our Hibernate configuration so that we
>>>>>>>>> are 1) properly using the open session in view pattern and 2)
>>>>>>>>> enabling lazy fetching on all objects and associations.
>>>>>>>>>
>>>>>>>>> Right now our Hibernate config is pretty messy and doesn't take
>>>>>>>>> advantage of many of Hibernate's performance features, so the
>>>>>>>>> main reason to do this work is to improve the performance of
>>>>>>>>> the business layer. The second big reason is just to reduce
>>>>>>>>> clutter and simplify the code as much as possible. There are
>>>>>>>>> plenty of places in the code where we have methods that aren't
>>>>>>>>> used at all or methods which are duplicated, so those would all
>>>>>>>>> be cleaned up.
>>>>>>>>>
>>>>>>>>> I have most of this work done already (but not checked in) and
>>>>>>>>> there aren't really any surprise changes that I had to make
>>>>>>>>> except when it came to the hierarchical objects. I tried for
>>>>>>>>> multiple days to get the hierarchical objects to work with the
>>>>>>>>> updated hibernate config and the current data model, but I kept
>>>>>>>>> running into problems. So to fix the problem I had to make a
>>>>>>>>> small tweak to the way hierarchical objects are persisted which
>>>>>>>>> fixed my issues and I believe drastically simplifies the
>>>>>>>>> problem overall. The basic change is that I have completely
>>>>>>>>> removed the HierarchicalPersistentObject class and Assoc and
>>>>>>>>> it's subclasses and changed the data model so that we have a
>>>>>>>>> more normal hierarchical model.
>>>>>>>>>
>>>>>>>>> So, for weblog categories I added a simple 'parentid' column to
>>>>>>>>> the weblogcategory table and that allows a category to manage
>>>>>>>>> relationships between it's parent and children directly. Same
>>>>>>>>> goes for the FolderData class, but as it turns out that column
>>>>>>>>> already existed in the schema but wasn't being used. Upgrade
>>>>>>>>> path for both of these is fairly simple and only requires
>>>>>>>>> populating these columns with the right value.
>>>>>>>>>
>>>>>>>>> I'm not sure if anyone really wants to see more of a proposal
>>>>>>>>> for this, which is why I started with an adhoc description here
>>>>>>>>> on the list. As I said, I am not actually modifying anything
>>>>>>>>> from a feature point of view, only cleaning up what is already
>>>>>>>>> there. If anyone wants to see more about the changes to the
>>>>>>>>> hierarchical objects then I can post them on the wiki or
>>>>>>>>> something.
>>>>>>>>>
>>>>>>>>> -- Allen
>>>>>>>>
>>>>>>>> Craig Russell
>>>>>>>> Architect, Sun Java Enterprise System
>>>>>>>> http://java.sun.com/products/jdo
>>>>>>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>>>>>>> P.S. A good JDO? O, Gasp!
>>>>>>>>
>>>>>>>
>>>>
>>>> Craig Russell
>>>> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
>>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>>> P.S. A good JDO? O, Gasp!
>>>>
>>>
>>
Re: Business layer cleanup for 3.2
Posted by Allen Gilliland <al...@sun.com>.
Anil Gangolli wrote:
>
> This actually has to do with the use of Hibernate Contextual Sessions (
> the thread-bound session returned by getCurrentSession() ), not
> Hibernate's behavior when using JDBC transaction semantics.
>
> http://www.hibernate.org/hib_docs/v3/reference/en/html/architecture.html#architecture-current-session
>
>
> We might want to consider using the ManagedSessionContext for this and
> managing the session binding/opening/closing ourselves; that might be
> cleaner and more localized than trying to manage flush() ourselves.
>
> The problem with managing flush() explicitly is that it tends to lead to
> having to assume boundaries in composition of operations (where the
> flush() is called). For example, if you query as part of a method you
> have to flush or know that flush is not necessary.
I am definitely interested in exploring this more. I have completed
what I originally set out to do and so far from my testing everything
seems to be working properly and the load testing that I have done has
shown some improvements here and there, but not all that I was hoping
for. Part of the reason there wasn't as big of a performance
improvement definitely may be due to the use of FlushMode.NEVER, so I
plan to investigate that as best I can and see if there are better
alternatives.
If you have any other ideas then definitely send them out and I will
attempt to try them out and see what kind of impact they have.
>
>
> Also, looking back at the initial message, I was confused by the phrase
> "2) enabling lazy fetching on all objects and
> associations" in Allen's message. Did you mean all single and
> collection-valued associations? I don't think lazy fetching of object
> properties will be a win at all.
Good point. Yes, I meant only for single and collection-valued
associations, which is what I believe Hibernate defaults to right now.
Now that I have everything working I plan to go back and do a bit more
rigorous debugging to double check this and make sure, but I am pretty
sure this is the case.
In any case, I would agree that lazy fetching for all properties would
be silly.
>
> We might also consider using join fetching for certain associations,
> particularly around weblog entry data. Maybe you looked at this already?
Nope, I haven't gotten that far. I am certainly open to this though, so
if you have time to offer up a few more details I can try experimenting
with it.
-- Allen
>
> --a.
>
>
> ----- Original Message ----- From: "Allen Gilliland"
> <al...@sun.com>
> To: <ro...@incubator.apache.org>
> Sent: Tuesday, November 14, 2006 11:16 AM
> Subject: Re: Business layer cleanup for 3.2
>
>
>>
>>
>> Craig L Russell wrote:
>>>
>>> On Nov 14, 2006, at 9:48 AM, Allen Gilliland wrote:
>>>
>>>>
>>>>
>>>>> It's not unlikely that we will break some things temporarily and
>>>>> not notice it for a while. The riskiest aspect to me in this
>>>>> regard is lazy fetching because it really demands that the session
>>>>> span the entire request, which we seemed to have a hard time doing
>>>>> properly earlier, and I'm not sure exactly why. We backed out of
>>>>> lazy fetching just before one release a ways back because we would
>>>>> hit odd session closed exceptions that we didn't have time to
>>>>> figure out. It's possible that some of Allen's earlier session
>>>>> management cleanups already got us out of those issues. It's a
>>>>> good idea to revisit this now. I think that is also likely to make
>>>>> us more portable to optimizations in other persistence
>>>>> implementations that expect essentially the same session management
>>>>> pattern.
>>>>
>>>> so far I have been able to make all the changes that I think are
>>>> correct and I have all the unit tests running correctly, so what I
>>>> am doing now is going over the actual webapp and running through all
>>>> the operations that I can to find and fix anything that I find.
>>>> Things are definitely cropping up, but so far the lazy
>>>> initialization problem hasn't come up.
>>>>
>>>> The bigger problem has been caused by changing the hibernate config
>>>> to use FlushMode.NEVER, which means that hibernate doesn't flush its
>>>> state to the db until we explicitly call the flush() method on the
>>>> Session. As it turns out, a *lot* of the stuff we were doing has
>>>> been very reliant on auto flushing for it to work, so there have
>>>> been a handful of places where I have had to figure out how to fix
>>>> that problem. So far so good though, and I hope to have things
>>>> cleaned up enough to commit in the next day or so.
>>>
>>> What is the reason you want to set hibernate config to use
>>> FlushMode.NEVER? From what I know of it, this is an antipattern.
>>
>> I will do some more reading about it because if it is an antipattern
>> then I should reconsider it, but the main reason why I am trying this
>> now is to get around some of Hibernates automation. The main problem
>> being that Hibernate's default handling for JDBCTransactions is setup
>> such that whenever you commit a transaction it closes the Session that
>> was handling that transaction, and that causes a problem with lazy
>> initialization because objects can no longer access associations if
>> the Session they were attached to is closed. I sifted through things
>> on the Hibernate forums to find other folks with the same problem and
>> how they fixed it and that's where I came up with FlushMode.NEVER.
>>
>> It may be possible that we can accomplish that without using
>> FlushMode.NEVER, but i'll have to look at it more carefully to see.
>>
>> Regardless of whether or not we use it I still think that in principal
>> it should work. The places where I have found things broken by this
>> change is situations where we are kind of circumventing our object
>> model and going directly to the db which causes a slight disconnect.
>> i.e. if I call saveObject(foo) and then later in the same transaction
>> try to getObject(foo) via a query before it has been flushed to the
>> db. So far these scenarios look more like slightly incorrect code as
>> much as anything else, but I am still investigating.
>>
>> I'll look into this more today though.
>>
>> -- Allen
>>
>>
>>>
>>> Craig
>>>>
>>>> -- Allen
>>>>
>>>>
>>>>> --a.
>>>>> ----- Original Message ----- From: "Allen Gilliland"
>>>>> <Al...@Sun.COM>
>>>>> To: <ro...@incubator.apache.org>
>>>>> Sent: Monday, November 13, 2006 4:46 PM
>>>>> Subject: Re: Business layer cleanup for 3.2
>>>>>> certainly. I didn't do much in the way of renaming things yet,
>>>>>> the first pass was mainly about fixing up the Hibernate config to
>>>>>> work the way I think it should have been working and clearing out
>>>>>> some things along the way. Once that is done then I plan to go
>>>>>> over the business layer more times and find places where methods
>>>>>> should be renamed, removed, or consolidated in any way. I also
>>>>>> want to keep building on the unit tests because I think they are
>>>>>> pretty good now, but there are a few gaps here and there.
>>>>>>
>>>>>> At the end of the day this work will definitely help to make the
>>>>>> work on the JDO/JPA backends quite a bit easier.
>>>>>>
>>>>>> -- Allen
>>>>>>
>>>>>>
>>>>>> Craig L Russell wrote:
>>>>>>> Hi Allen,
>>>>>>>
>>>>>>> We had discussed a number of issues with the manager classes such
>>>>>>> as misspelled method names and incomplete functionality (having
>>>>>>> the caller iterate through collections).
>>>>>>>
>>>>>>> I'd be happy to review what you've done in terms of cleanup.
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Craig
>>>>>>>
>>>>>>> On Nov 13, 2006, at 2:10 PM, Allen Gilliland wrote:
>>>>>>>
>>>>>>>> One of the things that I am planning to do for the 3.2 release
>>>>>>>> is do some audit/cleanup of the current business layer code.
>>>>>>>> There are a variety of things which could use improving, but the
>>>>>>>> main goal is to fix our Hibernate configuration so that we are
>>>>>>>> 1) properly using the open session in view pattern and 2)
>>>>>>>> enabling lazy fetching on all objects and associations.
>>>>>>>>
>>>>>>>> Right now our Hibernate config is pretty messy and doesn't take
>>>>>>>> advantage of many of Hibernate's performance features, so the
>>>>>>>> main reason to do this work is to improve the performance of the
>>>>>>>> business layer. The second big reason is just to reduce clutter
>>>>>>>> and simplify the code as much as possible. There are plenty of
>>>>>>>> places in the code where we have methods that aren't used at all
>>>>>>>> or methods which are duplicated, so those would all be cleaned up.
>>>>>>>>
>>>>>>>> I have most of this work done already (but not checked in) and
>>>>>>>> there aren't really any surprise changes that I had to make
>>>>>>>> except when it came to the hierarchical objects. I tried for
>>>>>>>> multiple days to get the hierarchical objects to work with the
>>>>>>>> updated hibernate config and the current data model, but I kept
>>>>>>>> running into problems. So to fix the problem I had to make a
>>>>>>>> small tweak to the way hierarchical objects are persisted which
>>>>>>>> fixed my issues and I believe drastically simplifies the problem
>>>>>>>> overall. The basic change is that I have completely removed the
>>>>>>>> HierarchicalPersistentObject class and Assoc and it's subclasses
>>>>>>>> and changed the data model so that we have a more normal
>>>>>>>> hierarchical model.
>>>>>>>>
>>>>>>>> So, for weblog categories I added a simple 'parentid' column to
>>>>>>>> the weblogcategory table and that allows a category to manage
>>>>>>>> relationships between it's parent and children directly. Same
>>>>>>>> goes for the FolderData class, but as it turns out that column
>>>>>>>> already existed in the schema but wasn't being used. Upgrade
>>>>>>>> path for both of these is fairly simple and only requires
>>>>>>>> populating these columns with the right value.
>>>>>>>>
>>>>>>>> I'm not sure if anyone really wants to see more of a proposal
>>>>>>>> for this, which is why I started with an adhoc description here
>>>>>>>> on the list. As I said, I am not actually modifying anything
>>>>>>>> from a feature point of view, only cleaning up what is already
>>>>>>>> there. If anyone wants to see more about the changes to the
>>>>>>>> hierarchical objects then I can post them on the wiki or something.
>>>>>>>>
>>>>>>>> -- Allen
>>>>>>>
>>>>>>> Craig Russell
>>>>>>> Architect, Sun Java Enterprise System
>>>>>>> http://java.sun.com/products/jdo
>>>>>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>>>>>> P.S. A good JDO? O, Gasp!
>>>>>>>
>>>>>>
>>>
>>> Craig Russell
>>> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>> P.S. A good JDO? O, Gasp!
>>>
>>
>
Re: Business layer cleanup for 3.2
Posted by Anil Gangolli <an...@busybuddha.org>.
This actually has to do with the use of Hibernate Contextual Sessions ( the
thread-bound session returned by getCurrentSession() ), not Hibernate's
behavior when using JDBC transaction semantics.
http://www.hibernate.org/hib_docs/v3/reference/en/html/architecture.html#architecture-current-session
We might want to consider using the ManagedSessionContext for this and
managing the session binding/opening/closing ourselves; that might be
cleaner and more localized than trying to manage flush() ourselves.
The problem with managing flush() explicitly is that it tends to lead to
having to assume boundaries in composition of operations (where the flush()
is called). For example, if you query as part of a method you have to flush
or know that flush is not necessary.
Also, looking back at the initial message, I was confused by the phrase "2)
enabling lazy fetching on all objects and
associations" in Allen's message. Did you mean all single and
collection-valued associations? I don't think lazy fetching of object
properties will be a win at all.
We might also consider using join fetching for certain associations,
particularly around weblog entry data. Maybe you looked at this already?
--a.
----- Original Message -----
From: "Allen Gilliland" <al...@sun.com>
To: <ro...@incubator.apache.org>
Sent: Tuesday, November 14, 2006 11:16 AM
Subject: Re: Business layer cleanup for 3.2
>
>
> Craig L Russell wrote:
>>
>> On Nov 14, 2006, at 9:48 AM, Allen Gilliland wrote:
>>
>>>
>>>
>>>> It's not unlikely that we will break some things temporarily and not
>>>> notice it for a while. The riskiest aspect to me in this regard is
>>>> lazy fetching because it really demands that the session span the
>>>> entire request, which we seemed to have a hard time doing properly
>>>> earlier, and I'm not sure exactly why. We backed out of lazy fetching
>>>> just before one release a ways back because we would hit odd session
>>>> closed exceptions that we didn't have time to figure out. It's
>>>> possible that some of Allen's earlier session management cleanups
>>>> already got us out of those issues. It's a good idea to revisit this
>>>> now. I think that is also likely to make us more portable to
>>>> optimizations in other persistence implementations that expect
>>>> essentially the same session management pattern.
>>>
>>> so far I have been able to make all the changes that I think are correct
>>> and I have all the unit tests running correctly, so what I am doing now
>>> is going over the actual webapp and running through all the operations
>>> that I can to find and fix anything that I find. Things are definitely
>>> cropping up, but so far the lazy initialization problem hasn't come up.
>>>
>>> The bigger problem has been caused by changing the hibernate config to
>>> use FlushMode.NEVER, which means that hibernate doesn't flush its state
>>> to the db until we explicitly call the flush() method on the Session. As
>>> it turns out, a *lot* of the stuff we were doing has been very reliant
>>> on auto flushing for it to work, so there have been a handful of places
>>> where I have had to figure out how to fix that problem. So far so good
>>> though, and I hope to have things cleaned up enough to commit in the
>>> next day or so.
>>
>> What is the reason you want to set hibernate config to use
>> FlushMode.NEVER? From what I know of it, this is an antipattern.
>
> I will do some more reading about it because if it is an antipattern then
> I should reconsider it, but the main reason why I am trying this now is to
> get around some of Hibernates automation. The main problem being that
> Hibernate's default handling for JDBCTransactions is setup such that
> whenever you commit a transaction it closes the Session that was handling
> that transaction, and that causes a problem with lazy initialization
> because objects can no longer access associations if the Session they were
> attached to is closed. I sifted through things on the Hibernate forums to
> find other folks with the same problem and how they fixed it and that's
> where I came up with FlushMode.NEVER.
>
> It may be possible that we can accomplish that without using
> FlushMode.NEVER, but i'll have to look at it more carefully to see.
>
> Regardless of whether or not we use it I still think that in principal it
> should work. The places where I have found things broken by this change
> is situations where we are kind of circumventing our object model and
> going directly to the db which causes a slight disconnect. i.e. if I call
> saveObject(foo) and then later in the same transaction try to
> getObject(foo) via a query before it has been flushed to the db. So far
> these scenarios look more like slightly incorrect code as much as anything
> else, but I am still investigating.
>
> I'll look into this more today though.
>
> -- Allen
>
>
>>
>> Craig
>>>
>>> -- Allen
>>>
>>>
>>>> --a.
>>>> ----- Original Message ----- From: "Allen Gilliland"
>>>> <Al...@Sun.COM>
>>>> To: <ro...@incubator.apache.org>
>>>> Sent: Monday, November 13, 2006 4:46 PM
>>>> Subject: Re: Business layer cleanup for 3.2
>>>>> certainly. I didn't do much in the way of renaming things yet, the
>>>>> first pass was mainly about fixing up the Hibernate config to work the
>>>>> way I think it should have been working and clearing out some things
>>>>> along the way. Once that is done then I plan to go over the business
>>>>> layer more times and find places where methods should be renamed,
>>>>> removed, or consolidated in any way. I also want to keep building on
>>>>> the unit tests because I think they are pretty good now, but there are
>>>>> a few gaps here and there.
>>>>>
>>>>> At the end of the day this work will definitely help to make the work
>>>>> on the JDO/JPA backends quite a bit easier.
>>>>>
>>>>> -- Allen
>>>>>
>>>>>
>>>>> Craig L Russell wrote:
>>>>>> Hi Allen,
>>>>>>
>>>>>> We had discussed a number of issues with the manager classes such as
>>>>>> misspelled method names and incomplete functionality (having the
>>>>>> caller iterate through collections).
>>>>>>
>>>>>> I'd be happy to review what you've done in terms of cleanup.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Craig
>>>>>>
>>>>>> On Nov 13, 2006, at 2:10 PM, Allen Gilliland wrote:
>>>>>>
>>>>>>> One of the things that I am planning to do for the 3.2 release is do
>>>>>>> some audit/cleanup of the current business layer code. There are a
>>>>>>> variety of things which could use improving, but the main goal is to
>>>>>>> fix our Hibernate configuration so that we are 1) properly using the
>>>>>>> open session in view pattern and 2) enabling lazy fetching on all
>>>>>>> objects and associations.
>>>>>>>
>>>>>>> Right now our Hibernate config is pretty messy and doesn't take
>>>>>>> advantage of many of Hibernate's performance features, so the main
>>>>>>> reason to do this work is to improve the performance of the business
>>>>>>> layer. The second big reason is just to reduce clutter and simplify
>>>>>>> the code as much as possible. There are plenty of places in the
>>>>>>> code where we have methods that aren't used at all or methods which
>>>>>>> are duplicated, so those would all be cleaned up.
>>>>>>>
>>>>>>> I have most of this work done already (but not checked in) and there
>>>>>>> aren't really any surprise changes that I had to make except when it
>>>>>>> came to the hierarchical objects. I tried for multiple days to get
>>>>>>> the hierarchical objects to work with the updated hibernate config
>>>>>>> and the current data model, but I kept running into problems. So to
>>>>>>> fix the problem I had to make a small tweak to the way hierarchical
>>>>>>> objects are persisted which fixed my issues and I believe
>>>>>>> drastically simplifies the problem overall. The basic change is
>>>>>>> that I have completely removed the HierarchicalPersistentObject
>>>>>>> class and Assoc and it's subclasses and changed the data model so
>>>>>>> that we have a more normal hierarchical model.
>>>>>>>
>>>>>>> So, for weblog categories I added a simple 'parentid' column to the
>>>>>>> weblogcategory table and that allows a category to manage
>>>>>>> relationships between it's parent and children directly. Same goes
>>>>>>> for the FolderData class, but as it turns out that column already
>>>>>>> existed in the schema but wasn't being used. Upgrade path for both
>>>>>>> of these is fairly simple and only requires populating these columns
>>>>>>> with the right value.
>>>>>>>
>>>>>>> I'm not sure if anyone really wants to see more of a proposal for
>>>>>>> this, which is why I started with an adhoc description here on the
>>>>>>> list. As I said, I am not actually modifying anything from a
>>>>>>> feature point of view, only cleaning up what is already there. If
>>>>>>> anyone wants to see more about the changes to the hierarchical
>>>>>>> objects then I can post them on the wiki or something.
>>>>>>>
>>>>>>> -- Allen
>>>>>>
>>>>>> Craig Russell
>>>>>> Architect, Sun Java Enterprise System
>>>>>> http://java.sun.com/products/jdo
>>>>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>>>>> P.S. A good JDO? O, Gasp!
>>>>>>
>>>>>
>>
>> Craig Russell
>> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
>> 408 276-5638 mailto:Craig.Russell@sun.com
>> P.S. A good JDO? O, Gasp!
>>
>
Re: Business layer cleanup for 3.2
Posted by Allen Gilliland <al...@sun.com>.
Craig L Russell wrote:
>
> On Nov 14, 2006, at 9:48 AM, Allen Gilliland wrote:
>
>>
>>
>>> It's not unlikely that we will break some things temporarily and not
>>> notice it for a while. The riskiest aspect to me in this regard is
>>> lazy fetching because it really demands that the session span the
>>> entire request, which we seemed to have a hard time doing properly
>>> earlier, and I'm not sure exactly why. We backed out of lazy
>>> fetching just before one release a ways back because we would hit odd
>>> session closed exceptions that we didn't have time to figure out.
>>> It's possible that some of Allen's earlier session management
>>> cleanups already got us out of those issues. It's a good idea to
>>> revisit this now. I think that is also likely to make us more
>>> portable to optimizations in other persistence implementations that
>>> expect essentially the same session management pattern.
>>
>> so far I have been able to make all the changes that I think are
>> correct and I have all the unit tests running correctly, so what I am
>> doing now is going over the actual webapp and running through all the
>> operations that I can to find and fix anything that I find. Things
>> are definitely cropping up, but so far the lazy initialization problem
>> hasn't come up.
>>
>> The bigger problem has been caused by changing the hibernate config to
>> use FlushMode.NEVER, which means that hibernate doesn't flush its
>> state to the db until we explicitly call the flush() method on the
>> Session. As it turns out, a *lot* of the stuff we were doing has been
>> very reliant on auto flushing for it to work, so there have been a
>> handful of places where I have had to figure out how to fix that
>> problem. So far so good though, and I hope to have things cleaned up
>> enough to commit in the next day or so.
>
> What is the reason you want to set hibernate config to use
> FlushMode.NEVER? From what I know of it, this is an antipattern.
I will do some more reading about it because if it is an antipattern
then I should reconsider it, but the main reason why I am trying this
now is to get around some of Hibernates automation. The main problem
being that Hibernate's default handling for JDBCTransactions is setup
such that whenever you commit a transaction it closes the Session that
was handling that transaction, and that causes a problem with lazy
initialization because objects can no longer access associations if the
Session they were attached to is closed. I sifted through things on the
Hibernate forums to find other folks with the same problem and how they
fixed it and that's where I came up with FlushMode.NEVER.
It may be possible that we can accomplish that without using
FlushMode.NEVER, but i'll have to look at it more carefully to see.
Regardless of whether or not we use it I still think that in principal
it should work. The places where I have found things broken by this
change is situations where we are kind of circumventing our object model
and going directly to the db which causes a slight disconnect. i.e. if
I call saveObject(foo) and then later in the same transaction try to
getObject(foo) via a query before it has been flushed to the db. So far
these scenarios look more like slightly incorrect code as much as
anything else, but I am still investigating.
I'll look into this more today though.
-- Allen
>
> Craig
>>
>> -- Allen
>>
>>
>>> --a.
>>> ----- Original Message ----- From: "Allen Gilliland"
>>> <Al...@Sun.COM>
>>> To: <ro...@incubator.apache.org>
>>> Sent: Monday, November 13, 2006 4:46 PM
>>> Subject: Re: Business layer cleanup for 3.2
>>>> certainly. I didn't do much in the way of renaming things yet, the
>>>> first pass was mainly about fixing up the Hibernate config to work
>>>> the way I think it should have been working and clearing out some
>>>> things along the way. Once that is done then I plan to go over the
>>>> business layer more times and find places where methods should be
>>>> renamed, removed, or consolidated in any way. I also want to keep
>>>> building on the unit tests because I think they are pretty good now,
>>>> but there are a few gaps here and there.
>>>>
>>>> At the end of the day this work will definitely help to make the
>>>> work on the JDO/JPA backends quite a bit easier.
>>>>
>>>> -- Allen
>>>>
>>>>
>>>> Craig L Russell wrote:
>>>>> Hi Allen,
>>>>>
>>>>> We had discussed a number of issues with the manager classes such
>>>>> as misspelled method names and incomplete functionality (having the
>>>>> caller iterate through collections).
>>>>>
>>>>> I'd be happy to review what you've done in terms of cleanup.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Craig
>>>>>
>>>>> On Nov 13, 2006, at 2:10 PM, Allen Gilliland wrote:
>>>>>
>>>>>> One of the things that I am planning to do for the 3.2 release is
>>>>>> do some audit/cleanup of the current business layer code. There
>>>>>> are a variety of things which could use improving, but the main
>>>>>> goal is to fix our Hibernate configuration so that we are 1)
>>>>>> properly using the open session in view pattern and 2) enabling
>>>>>> lazy fetching on all objects and associations.
>>>>>>
>>>>>> Right now our Hibernate config is pretty messy and doesn't take
>>>>>> advantage of many of Hibernate's performance features, so the main
>>>>>> reason to do this work is to improve the performance of the
>>>>>> business layer. The second big reason is just to reduce clutter
>>>>>> and simplify the code as much as possible. There are plenty of
>>>>>> places in the code where we have methods that aren't used at all
>>>>>> or methods which are duplicated, so those would all be cleaned up.
>>>>>>
>>>>>> I have most of this work done already (but not checked in) and
>>>>>> there aren't really any surprise changes that I had to make except
>>>>>> when it came to the hierarchical objects. I tried for multiple
>>>>>> days to get the hierarchical objects to work with the updated
>>>>>> hibernate config and the current data model, but I kept running
>>>>>> into problems. So to fix the problem I had to make a small tweak
>>>>>> to the way hierarchical objects are persisted which fixed my
>>>>>> issues and I believe drastically simplifies the problem overall.
>>>>>> The basic change is that I have completely removed the
>>>>>> HierarchicalPersistentObject class and Assoc and it's subclasses
>>>>>> and changed the data model so that we have a more normal
>>>>>> hierarchical model.
>>>>>>
>>>>>> So, for weblog categories I added a simple 'parentid' column to
>>>>>> the weblogcategory table and that allows a category to manage
>>>>>> relationships between it's parent and children directly. Same
>>>>>> goes for the FolderData class, but as it turns out that column
>>>>>> already existed in the schema but wasn't being used. Upgrade path
>>>>>> for both of these is fairly simple and only requires populating
>>>>>> these columns with the right value.
>>>>>>
>>>>>> I'm not sure if anyone really wants to see more of a proposal for
>>>>>> this, which is why I started with an adhoc description here on the
>>>>>> list. As I said, I am not actually modifying anything from a
>>>>>> feature point of view, only cleaning up what is already there. If
>>>>>> anyone wants to see more about the changes to the hierarchical
>>>>>> objects then I can post them on the wiki or something.
>>>>>>
>>>>>> -- Allen
>>>>>
>>>>> Craig Russell
>>>>> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
>>>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>>>> P.S. A good JDO? O, Gasp!
>>>>>
>>>>
>
> Craig Russell
> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
> 408 276-5638 mailto:Craig.Russell@sun.com
> P.S. A good JDO? O, Gasp!
>
Re: Business layer cleanup for 3.2
Posted by Craig L Russell <Cr...@Sun.COM>.
On Nov 14, 2006, at 9:48 AM, Allen Gilliland wrote:
>
>
> Anil Gangolli wrote:
>> +1 on the cleanup work in general. A big hoorah for the
>> hierarchical object cleanup!
>> I'd like to review the changes related to the persistence pattern
>> so that I understand them, but you can just let me know the commit
>> revisions involved.
>
> I could do this on the wiki, but it seems easy and short enough to
> just do it on the list. So, the most fundamental change is that
> the folderassoc and weblogcategoryassoc db tables are not used
> anymore, and the HierarchicalPO, Assoc, WeblogCategoryAssoc, and
> FolderAssoc classes are all being deleted along with any methods
> and code which referenced them.
>
> I'll just detail the weblog category example since it's the same
> for folders. So we are going to add a single column 'parentid' to
> the weblogcategory table which means the table will look like ...
>
> create table weblogcategory (
> id varchar(48) not null primary key,
> name varchar(255) not null,
> description varchar(255),
> websiteid varchar(48) not null,
> image varchar(255),
> parentid varchar(48)
> );
>
> By doing that there are 2 significant changes in the pojo because
> we can now directly reference the parent and children of a category
> using simple hibernate mappings ...
>
> /**
> * @hibernate.many-to-one column="parentid" cascade="none" not-
> null="false"
> */
> public WeblogCategoryData getParent() {
> return this.parentCategory;
> }
>
>
> /**
> * @hibernate.set lazy="true" inverse="true" cascade="delete"
> * @hibernate.collection-key column="parentid"
> * @hibernate.collection-one-to-many
> class="org.apache.roller.pojos.WeblogCategoryData"
> */
> public Set getWeblogCategories() {
> return this.childCategories;
> }
>
> So as you can see, the parent of a given category is a simply many-
> to-one association and of course for the root node the parent is
> NULL. Then for the child nodes it's a typical one-to-many
> association which simply points back to the same table and is
> basically formed by querying for all categories which list the
> current object as it's parent.
>
> That basically sums up all the changes as far as associations are
> concerned and provides an easy way to walk the tree. The funny
> thing was that many of the other methods which I thought would have
> needed more tweaking to work with these changes didn't really need
> much changing at all. Methods like retrieveEntries() or getPath()
> were already being formed by simply walking the tree and compiling
> the objects, so they didn't change too much at all.
>
>
>> If we make these changes, it's good to do it early in the release
>> cycle (as soon as 3.1 is branched off of main) so they get some
>> road time in development before we release.
>
> definitely.
>
>
>> It's not unlikely that we will break some things temporarily and
>> not notice it for a while. The riskiest aspect to me in this
>> regard is lazy fetching because it really demands that the session
>> span the entire request, which we seemed to have a hard time doing
>> properly earlier, and I'm not sure exactly why. We backed out of
>> lazy fetching just before one release a ways back because we would
>> hit odd session closed exceptions that we didn't have time to
>> figure out. It's possible that some of Allen's earlier session
>> management cleanups already got us out of those issues. It's a
>> good idea to revisit this now. I think that is also likely to
>> make us more portable to optimizations in other persistence
>> implementations that expect essentially the same session
>> management pattern.
>
> so far I have been able to make all the changes that I think are
> correct and I have all the unit tests running correctly, so what I
> am doing now is going over the actual webapp and running through
> all the operations that I can to find and fix anything that I
> find. Things are definitely cropping up, but so far the lazy
> initialization problem hasn't come up.
>
> The bigger problem has been caused by changing the hibernate config
> to use FlushMode.NEVER, which means that hibernate doesn't flush
> its state to the db until we explicitly call the flush() method on
> the Session. As it turns out, a *lot* of the stuff we were doing
> has been very reliant on auto flushing for it to work, so there
> have been a handful of places where I have had to figure out how to
> fix that problem. So far so good though, and I hope to have things
> cleaned up enough to commit in the next day or so.
What is the reason you want to set hibernate config to use
FlushMode.NEVER? From what I know of it, this is an antipattern.
Craig
>
> -- Allen
>
>
>> --a.
>> ----- Original Message ----- From: "Allen Gilliland"
>> <Al...@Sun.COM>
>> To: <ro...@incubator.apache.org>
>> Sent: Monday, November 13, 2006 4:46 PM
>> Subject: Re: Business layer cleanup for 3.2
>>> certainly. I didn't do much in the way of renaming things yet,
>>> the first pass was mainly about fixing up the Hibernate config to
>>> work the way I think it should have been working and clearing out
>>> some things along the way. Once that is done then I plan to go
>>> over the business layer more times and find places where methods
>>> should be renamed, removed, or consolidated in any way. I also
>>> want to keep building on the unit tests because I think they are
>>> pretty good now, but there are a few gaps here and there.
>>>
>>> At the end of the day this work will definitely help to make the
>>> work on the JDO/JPA backends quite a bit easier.
>>>
>>> -- Allen
>>>
>>>
>>> Craig L Russell wrote:
>>>> Hi Allen,
>>>>
>>>> We had discussed a number of issues with the manager classes
>>>> such as misspelled method names and incomplete functionality
>>>> (having the caller iterate through collections).
>>>>
>>>> I'd be happy to review what you've done in terms of cleanup.
>>>>
>>>> Regards,
>>>>
>>>> Craig
>>>>
>>>> On Nov 13, 2006, at 2:10 PM, Allen Gilliland wrote:
>>>>
>>>>> One of the things that I am planning to do for the 3.2 release
>>>>> is do some audit/cleanup of the current business layer code.
>>>>> There are a variety of things which could use improving, but
>>>>> the main goal is to fix our Hibernate configuration so that we
>>>>> are 1) properly using the open session in view pattern and 2)
>>>>> enabling lazy fetching on all objects and associations.
>>>>>
>>>>> Right now our Hibernate config is pretty messy and doesn't take
>>>>> advantage of many of Hibernate's performance features, so the
>>>>> main reason to do this work is to improve the performance of
>>>>> the business layer. The second big reason is just to reduce
>>>>> clutter and simplify the code as much as possible. There are
>>>>> plenty of places in the code where we have methods that aren't
>>>>> used at all or methods which are duplicated, so those would all
>>>>> be cleaned up.
>>>>>
>>>>> I have most of this work done already (but not checked in) and
>>>>> there aren't really any surprise changes that I had to make
>>>>> except when it came to the hierarchical objects. I tried for
>>>>> multiple days to get the hierarchical objects to work with the
>>>>> updated hibernate config and the current data model, but I kept
>>>>> running into problems. So to fix the problem I had to make a
>>>>> small tweak to the way hierarchical objects are persisted which
>>>>> fixed my issues and I believe drastically simplifies the
>>>>> problem overall. The basic change is that I have completely
>>>>> removed the HierarchicalPersistentObject class and Assoc and
>>>>> it's subclasses and changed the data model so that we have a
>>>>> more normal hierarchical model.
>>>>>
>>>>> So, for weblog categories I added a simple 'parentid' column to
>>>>> the weblogcategory table and that allows a category to manage
>>>>> relationships between it's parent and children directly. Same
>>>>> goes for the FolderData class, but as it turns out that column
>>>>> already existed in the schema but wasn't being used. Upgrade
>>>>> path for both of these is fairly simple and only requires
>>>>> populating these columns with the right value.
>>>>>
>>>>> I'm not sure if anyone really wants to see more of a proposal
>>>>> for this, which is why I started with an adhoc description here
>>>>> on the list. As I said, I am not actually modifying anything
>>>>> from a feature point of view, only cleaning up what is already
>>>>> there. If anyone wants to see more about the changes to the
>>>>> hierarchical objects then I can post them on the wiki or
>>>>> something.
>>>>>
>>>>> -- Allen
>>>>
>>>> Craig Russell
>>>> Architect, Sun Java Enterprise System http://java.sun.com/
>>>> products/jdo
>>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>>> P.S. A good JDO? O, Gasp!
>>>>
>>>
Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
408 276-5638 mailto:Craig.Russell@sun.com
P.S. A good JDO? O, Gasp!
Re: Business layer cleanup for 3.2
Posted by Allen Gilliland <al...@sun.com>.
Okay, I have a little bit more testing that I plan to do, but I think I
am actually very close to having this ready to be committed so I wanted
to give a final heads up and see if there is still anyone who wants to
know more about what I'm changing before I commit. The list of changed
files is here ...
M src/org/apache/roller/pojos/BookmarkData.java
M src/org/apache/roller/pojos/FolderData.java
D src/org/apache/roller/pojos/WeblogCategoryAssoc.java
D src/org/apache/roller/pojos/FolderAssoc.java
M src/org/apache/roller/pojos/WeblogCategoryData.java
D src/org/apache/roller/pojos/Assoc.java
D src/org/apache/roller/pojos/HierarchicalPersistentObject.java
M src/org/apache/roller/pojos/WebsiteData.java
M src/org/apache/roller/business/BookmarkManager.java
M src/org/apache/roller/business/WeblogManager.java
M src/org/apache/roller/business/ThemeManagerImpl.java
A
src/org/apache/roller/business/hibernate/ThreadLocalSessionContextNoAutoClose.java
M
src/org/apache/roller/business/hibernate/HibernateBookmarkManagerImpl.java
M
src/org/apache/roller/business/hibernate/HibernatePersistenceStrategy.java
M
src/org/apache/roller/business/hibernate/HibernateUserManagerImpl.java
M
src/org/apache/roller/business/hibernate/HibernateWeblogManagerImpl.java
M
src/org/apache/roller/ui/authoring/struts/formbeans/WeblogCategoryFormEx.java
M src/org/apache/roller/ui/core/RollerSession.java
M src/org/apache/roller/ui/rendering/plugins/BookmarkPlugin.java
M
src/org/apache/roller/ui/rendering/velocity/deprecated/OldWeblogPageModel.java
most of the changes are in the folders and categories pojos and their
respective manager methods. the rest of the changes are just tweaks
needed as a result of the changes to folders and cats and some cleanup
in the hibernate strategy class.
I am thinking about committing this later this afternoon.
-- Allen
Allen Gilliland wrote:
>
>
> Anil Gangolli wrote:
>>
>> +1 on the cleanup work in general. A big hoorah for the hierarchical
>> object cleanup!
>>
>> I'd like to review the changes related to the persistence pattern so
>> that I understand them, but you can just let me know the commit
>> revisions involved.
>
> I could do this on the wiki, but it seems easy and short enough to just
> do it on the list. So, the most fundamental change is that the
> folderassoc and weblogcategoryassoc db tables are not used anymore, and
> the HierarchicalPO, Assoc, WeblogCategoryAssoc, and FolderAssoc classes
> are all being deleted along with any methods and code which referenced
> them.
>
> I'll just detail the weblog category example since it's the same for
> folders. So we are going to add a single column 'parentid' to the
> weblogcategory table which means the table will look like ...
>
> create table weblogcategory (
> id varchar(48) not null primary key,
> name varchar(255) not null,
> description varchar(255),
> websiteid varchar(48) not null,
> image varchar(255),
> parentid varchar(48)
> );
>
> By doing that there are 2 significant changes in the pojo because we can
> now directly reference the parent and children of a category using
> simple hibernate mappings ...
>
> /**
> * @hibernate.many-to-one column="parentid" cascade="none"
> not-null="false"
> */
> public WeblogCategoryData getParent() {
> return this.parentCategory;
> }
>
>
> /**
> * @hibernate.set lazy="true" inverse="true" cascade="delete"
> * @hibernate.collection-key column="parentid"
> * @hibernate.collection-one-to-many
> class="org.apache.roller.pojos.WeblogCategoryData"
> */
> public Set getWeblogCategories() {
> return this.childCategories;
> }
>
> So as you can see, the parent of a given category is a simply
> many-to-one association and of course for the root node the parent is
> NULL. Then for the child nodes it's a typical one-to-many association
> which simply points back to the same table and is basically formed by
> querying for all categories which list the current object as it's parent.
>
> That basically sums up all the changes as far as associations are
> concerned and provides an easy way to walk the tree. The funny thing
> was that many of the other methods which I thought would have needed
> more tweaking to work with these changes didn't really need much
> changing at all. Methods like retrieveEntries() or getPath() were
> already being formed by simply walking the tree and compiling the
> objects, so they didn't change too much at all.
>
>
>>
>> If we make these changes, it's good to do it early in the release
>> cycle (as soon as 3.1 is branched off of main) so they get some road
>> time in development before we release.
>
> definitely.
>
>
>>
>> It's not unlikely that we will break some things temporarily and not
>> notice it for a while. The riskiest aspect to me in this regard is
>> lazy fetching because it really demands that the session span the
>> entire request, which we seemed to have a hard time doing properly
>> earlier, and I'm not sure exactly why. We backed out of lazy fetching
>> just before one release a ways back because we would hit odd session
>> closed exceptions that we didn't have time to figure out. It's
>> possible that some of Allen's earlier session management cleanups
>> already got us out of those issues. It's a good idea to revisit this
>> now. I think that is also likely to make us more portable to
>> optimizations in other persistence implementations that expect
>> essentially the same session management pattern.
>
> so far I have been able to make all the changes that I think are correct
> and I have all the unit tests running correctly, so what I am doing now
> is going over the actual webapp and running through all the operations
> that I can to find and fix anything that I find. Things are definitely
> cropping up, but so far the lazy initialization problem hasn't come up.
>
> The bigger problem has been caused by changing the hibernate config to
> use FlushMode.NEVER, which means that hibernate doesn't flush its state
> to the db until we explicitly call the flush() method on the Session. As
> it turns out, a *lot* of the stuff we were doing has been very reliant
> on auto flushing for it to work, so there have been a handful of places
> where I have had to figure out how to fix that problem. So far so good
> though, and I hope to have things cleaned up enough to commit in the
> next day or so.
>
> -- Allen
>
>
>>
>> --a.
>>
>>
>>
>> ----- Original Message ----- From: "Allen Gilliland"
>> <Al...@Sun.COM>
>> To: <ro...@incubator.apache.org>
>> Sent: Monday, November 13, 2006 4:46 PM
>> Subject: Re: Business layer cleanup for 3.2
>>
>>
>>> certainly. I didn't do much in the way of renaming things yet, the
>>> first pass was mainly about fixing up the Hibernate config to work
>>> the way I think it should have been working and clearing out some
>>> things along the way. Once that is done then I plan to go over the
>>> business layer more times and find places where methods should be
>>> renamed, removed, or consolidated in any way. I also want to keep
>>> building on the unit tests because I think they are pretty good now,
>>> but there are a few gaps here and there.
>>>
>>> At the end of the day this work will definitely help to make the work
>>> on the JDO/JPA backends quite a bit easier.
>>>
>>> -- Allen
>>>
>>>
>>> Craig L Russell wrote:
>>>> Hi Allen,
>>>>
>>>> We had discussed a number of issues with the manager classes such as
>>>> misspelled method names and incomplete functionality (having the
>>>> caller iterate through collections).
>>>>
>>>> I'd be happy to review what you've done in terms of cleanup.
>>>>
>>>> Regards,
>>>>
>>>> Craig
>>>>
>>>> On Nov 13, 2006, at 2:10 PM, Allen Gilliland wrote:
>>>>
>>>>> One of the things that I am planning to do for the 3.2 release is
>>>>> do some audit/cleanup of the current business layer code. There
>>>>> are a variety of things which could use improving, but the main
>>>>> goal is to fix our Hibernate configuration so that we are 1)
>>>>> properly using the open session in view pattern and 2) enabling
>>>>> lazy fetching on all objects and associations.
>>>>>
>>>>> Right now our Hibernate config is pretty messy and doesn't take
>>>>> advantage of many of Hibernate's performance features, so the main
>>>>> reason to do this work is to improve the performance of the
>>>>> business layer. The second big reason is just to reduce clutter
>>>>> and simplify the code as much as possible. There are plenty of
>>>>> places in the code where we have methods that aren't used at all or
>>>>> methods which are duplicated, so those would all be cleaned up.
>>>>>
>>>>> I have most of this work done already (but not checked in) and
>>>>> there aren't really any surprise changes that I had to make except
>>>>> when it came to the hierarchical objects. I tried for multiple
>>>>> days to get the hierarchical objects to work with the updated
>>>>> hibernate config and the current data model, but I kept running
>>>>> into problems. So to fix the problem I had to make a small tweak
>>>>> to the way hierarchical objects are persisted which fixed my issues
>>>>> and I believe drastically simplifies the problem overall. The
>>>>> basic change is that I have completely removed the
>>>>> HierarchicalPersistentObject class and Assoc and it's subclasses
>>>>> and changed the data model so that we have a more normal
>>>>> hierarchical model.
>>>>>
>>>>> So, for weblog categories I added a simple 'parentid' column to the
>>>>> weblogcategory table and that allows a category to manage
>>>>> relationships between it's parent and children directly. Same goes
>>>>> for the FolderData class, but as it turns out that column already
>>>>> existed in the schema but wasn't being used. Upgrade path for both
>>>>> of these is fairly simple and only requires populating these
>>>>> columns with the right value.
>>>>>
>>>>> I'm not sure if anyone really wants to see more of a proposal for
>>>>> this, which is why I started with an adhoc description here on the
>>>>> list. As I said, I am not actually modifying anything from a
>>>>> feature point of view, only cleaning up what is already there. If
>>>>> anyone wants to see more about the changes to the hierarchical
>>>>> objects then I can post them on the wiki or something.
>>>>>
>>>>> -- Allen
>>>>
>>>> Craig Russell
>>>> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
>>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>>> P.S. A good JDO? O, Gasp!
>>>>
>>>
>>
Re: Business layer cleanup for 3.2
Posted by Allen Gilliland <al...@sun.com>.
Anil Gangolli wrote:
>
> +1 on the cleanup work in general. A big hoorah for the hierarchical
> object cleanup!
>
> I'd like to review the changes related to the persistence pattern so
> that I understand them, but you can just let me know the commit
> revisions involved.
I could do this on the wiki, but it seems easy and short enough to just
do it on the list. So, the most fundamental change is that the
folderassoc and weblogcategoryassoc db tables are not used anymore, and
the HierarchicalPO, Assoc, WeblogCategoryAssoc, and FolderAssoc classes
are all being deleted along with any methods and code which referenced them.
I'll just detail the weblog category example since it's the same for
folders. So we are going to add a single column 'parentid' to the
weblogcategory table which means the table will look like ...
create table weblogcategory (
id varchar(48) not null primary key,
name varchar(255) not null,
description varchar(255),
websiteid varchar(48) not null,
image varchar(255),
parentid varchar(48)
);
By doing that there are 2 significant changes in the pojo because we can
now directly reference the parent and children of a category using
simple hibernate mappings ...
/**
* @hibernate.many-to-one column="parentid" cascade="none"
not-null="false"
*/
public WeblogCategoryData getParent() {
return this.parentCategory;
}
/**
* @hibernate.set lazy="true" inverse="true" cascade="delete"
* @hibernate.collection-key column="parentid"
* @hibernate.collection-one-to-many
class="org.apache.roller.pojos.WeblogCategoryData"
*/
public Set getWeblogCategories() {
return this.childCategories;
}
So as you can see, the parent of a given category is a simply
many-to-one association and of course for the root node the parent is
NULL. Then for the child nodes it's a typical one-to-many association
which simply points back to the same table and is basically formed by
querying for all categories which list the current object as it's parent.
That basically sums up all the changes as far as associations are
concerned and provides an easy way to walk the tree. The funny thing
was that many of the other methods which I thought would have needed
more tweaking to work with these changes didn't really need much
changing at all. Methods like retrieveEntries() or getPath() were
already being formed by simply walking the tree and compiling the
objects, so they didn't change too much at all.
>
> If we make these changes, it's good to do it early in the release cycle
> (as soon as 3.1 is branched off of main) so they get some road time in
> development before we release.
definitely.
>
> It's not unlikely that we will break some things temporarily and not
> notice it for a while. The riskiest aspect to me in this regard is lazy
> fetching because it really demands that the session span the entire
> request, which we seemed to have a hard time doing properly earlier, and
> I'm not sure exactly why. We backed out of lazy fetching just before
> one release a ways back because we would hit odd session closed
> exceptions that we didn't have time to figure out. It's possible that
> some of Allen's earlier session management cleanups already got us out
> of those issues. It's a good idea to revisit this now. I think that is
> also likely to make us more portable to optimizations in other
> persistence implementations that expect essentially the same session
> management pattern.
so far I have been able to make all the changes that I think are correct
and I have all the unit tests running correctly, so what I am doing now
is going over the actual webapp and running through all the operations
that I can to find and fix anything that I find. Things are definitely
cropping up, but so far the lazy initialization problem hasn't come up.
The bigger problem has been caused by changing the hibernate config to
use FlushMode.NEVER, which means that hibernate doesn't flush its state
to the db until we explicitly call the flush() method on the Session.
As it turns out, a *lot* of the stuff we were doing has been very
reliant on auto flushing for it to work, so there have been a handful of
places where I have had to figure out how to fix that problem. So far
so good though, and I hope to have things cleaned up enough to commit in
the next day or so.
-- Allen
>
> --a.
>
>
>
> ----- Original Message ----- From: "Allen Gilliland"
> <Al...@Sun.COM>
> To: <ro...@incubator.apache.org>
> Sent: Monday, November 13, 2006 4:46 PM
> Subject: Re: Business layer cleanup for 3.2
>
>
>> certainly. I didn't do much in the way of renaming things yet, the
>> first pass was mainly about fixing up the Hibernate config to work the
>> way I think it should have been working and clearing out some things
>> along the way. Once that is done then I plan to go over the business
>> layer more times and find places where methods should be renamed,
>> removed, or consolidated in any way. I also want to keep building on
>> the unit tests because I think they are pretty good now, but there are
>> a few gaps here and there.
>>
>> At the end of the day this work will definitely help to make the work
>> on the JDO/JPA backends quite a bit easier.
>>
>> -- Allen
>>
>>
>> Craig L Russell wrote:
>>> Hi Allen,
>>>
>>> We had discussed a number of issues with the manager classes such as
>>> misspelled method names and incomplete functionality (having the
>>> caller iterate through collections).
>>>
>>> I'd be happy to review what you've done in terms of cleanup.
>>>
>>> Regards,
>>>
>>> Craig
>>>
>>> On Nov 13, 2006, at 2:10 PM, Allen Gilliland wrote:
>>>
>>>> One of the things that I am planning to do for the 3.2 release is do
>>>> some audit/cleanup of the current business layer code. There are a
>>>> variety of things which could use improving, but the main goal is to
>>>> fix our Hibernate configuration so that we are 1) properly using the
>>>> open session in view pattern and 2) enabling lazy fetching on all
>>>> objects and associations.
>>>>
>>>> Right now our Hibernate config is pretty messy and doesn't take
>>>> advantage of many of Hibernate's performance features, so the main
>>>> reason to do this work is to improve the performance of the business
>>>> layer. The second big reason is just to reduce clutter and simplify
>>>> the code as much as possible. There are plenty of places in the
>>>> code where we have methods that aren't used at all or methods which
>>>> are duplicated, so those would all be cleaned up.
>>>>
>>>> I have most of this work done already (but not checked in) and there
>>>> aren't really any surprise changes that I had to make except when it
>>>> came to the hierarchical objects. I tried for multiple days to get
>>>> the hierarchical objects to work with the updated hibernate config
>>>> and the current data model, but I kept running into problems. So to
>>>> fix the problem I had to make a small tweak to the way hierarchical
>>>> objects are persisted which fixed my issues and I believe
>>>> drastically simplifies the problem overall. The basic change is
>>>> that I have completely removed the HierarchicalPersistentObject
>>>> class and Assoc and it's subclasses and changed the data model so
>>>> that we have a more normal hierarchical model.
>>>>
>>>> So, for weblog categories I added a simple 'parentid' column to the
>>>> weblogcategory table and that allows a category to manage
>>>> relationships between it's parent and children directly. Same goes
>>>> for the FolderData class, but as it turns out that column already
>>>> existed in the schema but wasn't being used. Upgrade path for both
>>>> of these is fairly simple and only requires populating these columns
>>>> with the right value.
>>>>
>>>> I'm not sure if anyone really wants to see more of a proposal for
>>>> this, which is why I started with an adhoc description here on the
>>>> list. As I said, I am not actually modifying anything from a
>>>> feature point of view, only cleaning up what is already there. If
>>>> anyone wants to see more about the changes to the hierarchical
>>>> objects then I can post them on the wiki or something.
>>>>
>>>> -- Allen
>>>
>>> Craig Russell
>>> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
>>> 408 276-5638 mailto:Craig.Russell@sun.com
>>> P.S. A good JDO? O, Gasp!
>>>
>>
>
Re: Business layer cleanup for 3.2
Posted by Anil Gangolli <an...@busybuddha.org>.
+1 on the cleanup work in general. A big hoorah for the hierarchical object
cleanup!
I'd like to review the changes related to the persistence pattern so that I
understand them, but you can just let me know the commit revisions involved.
If we make these changes, it's good to do it early in the release cycle (as
soon as 3.1 is branched off of main) so they get some road time in
development before we release.
It's not unlikely that we will break some things temporarily and not notice
it for a while. The riskiest aspect to me in this regard is lazy fetching
because it really demands that the session span the entire request, which we
seemed to have a hard time doing properly earlier, and I'm not sure exactly
why. We backed out of lazy fetching just before one release a ways back
because we would hit odd session closed exceptions that we didn't have time
to figure out. It's possible that some of Allen's earlier session
management cleanups already got us out of those issues. It's a good idea to
revisit this now. I think that is also likely to make us more portable to
optimizations in other persistence implementations that expect essentially
the same session management pattern.
--a.
----- Original Message -----
From: "Allen Gilliland" <Al...@Sun.COM>
To: <ro...@incubator.apache.org>
Sent: Monday, November 13, 2006 4:46 PM
Subject: Re: Business layer cleanup for 3.2
> certainly. I didn't do much in the way of renaming things yet, the first
> pass was mainly about fixing up the Hibernate config to work the way I
> think it should have been working and clearing out some things along the
> way. Once that is done then I plan to go over the business layer more
> times and find places where methods should be renamed, removed, or
> consolidated in any way. I also want to keep building on the unit tests
> because I think they are pretty good now, but there are a few gaps here
> and there.
>
> At the end of the day this work will definitely help to make the work on
> the JDO/JPA backends quite a bit easier.
>
> -- Allen
>
>
> Craig L Russell wrote:
>> Hi Allen,
>>
>> We had discussed a number of issues with the manager classes such as
>> misspelled method names and incomplete functionality (having the caller
>> iterate through collections).
>>
>> I'd be happy to review what you've done in terms of cleanup.
>>
>> Regards,
>>
>> Craig
>>
>> On Nov 13, 2006, at 2:10 PM, Allen Gilliland wrote:
>>
>>> One of the things that I am planning to do for the 3.2 release is do
>>> some audit/cleanup of the current business layer code. There are a
>>> variety of things which could use improving, but the main goal is to fix
>>> our Hibernate configuration so that we are 1) properly using the open
>>> session in view pattern and 2) enabling lazy fetching on all objects and
>>> associations.
>>>
>>> Right now our Hibernate config is pretty messy and doesn't take
>>> advantage of many of Hibernate's performance features, so the main
>>> reason to do this work is to improve the performance of the business
>>> layer. The second big reason is just to reduce clutter and simplify the
>>> code as much as possible. There are plenty of places in the code where
>>> we have methods that aren't used at all or methods which are duplicated,
>>> so those would all be cleaned up.
>>>
>>> I have most of this work done already (but not checked in) and there
>>> aren't really any surprise changes that I had to make except when it
>>> came to the hierarchical objects. I tried for multiple days to get the
>>> hierarchical objects to work with the updated hibernate config and the
>>> current data model, but I kept running into problems. So to fix the
>>> problem I had to make a small tweak to the way hierarchical objects are
>>> persisted which fixed my issues and I believe drastically simplifies the
>>> problem overall. The basic change is that I have completely removed the
>>> HierarchicalPersistentObject class and Assoc and it's subclasses and
>>> changed the data model so that we have a more normal hierarchical model.
>>>
>>> So, for weblog categories I added a simple 'parentid' column to the
>>> weblogcategory table and that allows a category to manage relationships
>>> between it's parent and children directly. Same goes for the FolderData
>>> class, but as it turns out that column already existed in the schema but
>>> wasn't being used. Upgrade path for both of these is fairly simple and
>>> only requires populating these columns with the right value.
>>>
>>> I'm not sure if anyone really wants to see more of a proposal for this,
>>> which is why I started with an adhoc description here on the list. As I
>>> said, I am not actually modifying anything from a feature point of view,
>>> only cleaning up what is already there. If anyone wants to see more
>>> about the changes to the hierarchical objects then I can post them on
>>> the wiki or something.
>>>
>>> -- Allen
>>
>> Craig Russell
>> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
>> 408 276-5638 mailto:Craig.Russell@sun.com
>> P.S. A good JDO? O, Gasp!
>>
>
Re: Business layer cleanup for 3.2
Posted by Allen Gilliland <Al...@Sun.COM>.
certainly. I didn't do much in the way of renaming things yet, the
first pass was mainly about fixing up the Hibernate config to work the
way I think it should have been working and clearing out some things
along the way. Once that is done then I plan to go over the business
layer more times and find places where methods should be renamed,
removed, or consolidated in any way. I also want to keep building on
the unit tests because I think they are pretty good now, but there are a
few gaps here and there.
At the end of the day this work will definitely help to make the work on
the JDO/JPA backends quite a bit easier.
-- Allen
Craig L Russell wrote:
> Hi Allen,
>
> We had discussed a number of issues with the manager classes such as
> misspelled method names and incomplete functionality (having the caller
> iterate through collections).
>
> I'd be happy to review what you've done in terms of cleanup.
>
> Regards,
>
> Craig
>
> On Nov 13, 2006, at 2:10 PM, Allen Gilliland wrote:
>
>> One of the things that I am planning to do for the 3.2 release is do
>> some audit/cleanup of the current business layer code. There are a
>> variety of things which could use improving, but the main goal is to
>> fix our Hibernate configuration so that we are 1) properly using the
>> open session in view pattern and 2) enabling lazy fetching on all
>> objects and associations.
>>
>> Right now our Hibernate config is pretty messy and doesn't take
>> advantage of many of Hibernate's performance features, so the main
>> reason to do this work is to improve the performance of the business
>> layer. The second big reason is just to reduce clutter and simplify
>> the code as much as possible. There are plenty of places in the code
>> where we have methods that aren't used at all or methods which are
>> duplicated, so those would all be cleaned up.
>>
>> I have most of this work done already (but not checked in) and there
>> aren't really any surprise changes that I had to make except when it
>> came to the hierarchical objects. I tried for multiple days to get
>> the hierarchical objects to work with the updated hibernate config and
>> the current data model, but I kept running into problems. So to fix
>> the problem I had to make a small tweak to the way hierarchical
>> objects are persisted which fixed my issues and I believe drastically
>> simplifies the problem overall. The basic change is that I have
>> completely removed the HierarchicalPersistentObject class and Assoc
>> and it's subclasses and changed the data model so that we have a more
>> normal hierarchical model.
>>
>> So, for weblog categories I added a simple 'parentid' column to the
>> weblogcategory table and that allows a category to manage
>> relationships between it's parent and children directly. Same goes
>> for the FolderData class, but as it turns out that column already
>> existed in the schema but wasn't being used. Upgrade path for both of
>> these is fairly simple and only requires populating these columns with
>> the right value.
>>
>> I'm not sure if anyone really wants to see more of a proposal for
>> this, which is why I started with an adhoc description here on the
>> list. As I said, I am not actually modifying anything from a feature
>> point of view, only cleaning up what is already there. If anyone
>> wants to see more about the changes to the hierarchical objects then I
>> can post them on the wiki or something.
>>
>> -- Allen
>
> Craig Russell
> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
> 408 276-5638 mailto:Craig.Russell@sun.com
> P.S. A good JDO? O, Gasp!
>
Re: Business layer cleanup for 3.2
Posted by Craig L Russell <Cr...@Sun.COM>.
Hi Allen,
We had discussed a number of issues with the manager classes such as
misspelled method names and incomplete functionality (having the
caller iterate through collections).
I'd be happy to review what you've done in terms of cleanup.
Regards,
Craig
On Nov 13, 2006, at 2:10 PM, Allen Gilliland wrote:
> One of the things that I am planning to do for the 3.2 release is
> do some audit/cleanup of the current business layer code. There
> are a variety of things which could use improving, but the main
> goal is to fix our Hibernate configuration so that we are 1)
> properly using the open session in view pattern and 2) enabling
> lazy fetching on all objects and associations.
>
> Right now our Hibernate config is pretty messy and doesn't take
> advantage of many of Hibernate's performance features, so the main
> reason to do this work is to improve the performance of the
> business layer. The second big reason is just to reduce clutter
> and simplify the code as much as possible. There are plenty of
> places in the code where we have methods that aren't used at all or
> methods which are duplicated, so those would all be cleaned up.
>
> I have most of this work done already (but not checked in) and
> there aren't really any surprise changes that I had to make except
> when it came to the hierarchical objects. I tried for multiple
> days to get the hierarchical objects to work with the updated
> hibernate config and the current data model, but I kept running
> into problems. So to fix the problem I had to make a small tweak
> to the way hierarchical objects are persisted which fixed my issues
> and I believe drastically simplifies the problem overall. The
> basic change is that I have completely removed the
> HierarchicalPersistentObject class and Assoc and it's subclasses
> and changed the data model so that we have a more normal
> hierarchical model.
>
> So, for weblog categories I added a simple 'parentid' column to the
> weblogcategory table and that allows a category to manage
> relationships between it's parent and children directly. Same goes
> for the FolderData class, but as it turns out that column already
> existed in the schema but wasn't being used. Upgrade path for both
> of these is fairly simple and only requires populating these
> columns with the right value.
>
> I'm not sure if anyone really wants to see more of a proposal for
> this, which is why I started with an adhoc description here on the
> list. As I said, I am not actually modifying anything from a
> feature point of view, only cleaning up what is already there. If
> anyone wants to see more about the changes to the hierarchical
> objects then I can post them on the wiki or something.
>
> -- Allen
Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
408 276-5638 mailto:Craig.Russell@sun.com
P.S. A good JDO? O, Gasp!
Re: Business layer cleanup for 3.2
Posted by Dave Levy <Da...@Sun.COM>.
Two points, see below
Craig L Russell wrote:
> Hi Allen,
>
> On Nov 14, 2006, at 9:06 AM, Allen Gilliland wrote:
>
>>
>>
>> Dave wrote:
>>> On 11/13/06, Allen Gilliland <al...@sun.com> wrote:
>>>> One of the things that I am planning to do for the 3.2 release is do
>>>> some audit/cleanup of the current business layer code. There are a
>>>> variety of things which could use improving, but the main goal is
>>>> to fix
>>>> our Hibernate configuration so that we are 1) properly using the open
>>>> session in view pattern and 2) enabling lazy fetching on all
>>>> objects and
>>>> associations.
>>> All good.
>>>> Right now our Hibernate config is pretty messy and doesn't take
>>>> advantage of many of Hibernate's performance features, so the main
>>>> reason to do this work is to improve the performance of the business
>>>> layer. The second big reason is just to reduce clutter and
>>>> simplify the
>>>> code as much as possible. There are plenty of places in the code
>>>> where
>>>> we have methods that aren't used at all or methods which are
>>>> duplicated,
>>>> so those would all be cleaned up.
>>>>
>>>> I have most of this work done already (but not checked in) and there
>>>> aren't really any surprise changes that I had to make except when it
>>>> came to the hierarchical objects. I tried for multiple days to get
>>>> the
>>>> hierarchical objects to work with the updated hibernate config and the
>>>> current data model, but I kept running into problems. So to fix the
>>>> problem I had to make a small tweak to the way hierarchical objects
>>>> are
>>>> persisted which fixed my issues and I believe drastically
>>>> simplifies the
>>>> problem overall. The basic change is that I have completely
>>>> removed the
>>>> HierarchicalPersistentObject class and Assoc and it's subclasses and
>>>> changed the data model so that we have a more normal hierarchical
>>>> model.
>>>>
>>>> So, for weblog categories I added a simple 'parentid' column to the
>>>> weblogcategory table and that allows a category to manage
>>>> relationships
>>>> between it's parent and children directly. Same goes for the
>>>> FolderData
>>>> class, but as it turns out that column already existed in the
>>>> schema but
>>>> wasn't being used. Upgrade path for both of these is fairly simple
>>>> and
>>>> only requires populating these columns with the right value.
>>> The hierarchical persistent object and the assoc tables were there for
>>> one purpose: to make it possible to do a recursive query (i.e. query
>>> for all entries under /pets and /pets/dogs, /pets/cats etc.) with only
>>> one SQL statement. But that hasn't worked since we made the switch to
>>> the Hibernate Criteria API back in 2004 and we've been doing fine
>>> without it. And the code that started as complex has been moved around
>>> so much that now it's almost unintelligible.
>>> So I'm +1 with that change.
>>
>> Actually, I have an idea that I think can accomplish what's described
>> above and actually kill a couple other birds with the same stone.
>> Basically, if we add another column for the full path of a node like
>> you did above (/pets, /pets/dogs, /pets/cats) then to get all entries
>> under the category /pets by getting all entries that have a category
>> with a path that starts with "/pets". So the sql query would just be
>> ...
>>
>> select * from weblogentry,weblogcategory where weblogentry.categoryid
>> = weblogcategory.id and weblogcategory.path like '/pets%';
Brilliant, never seen this before but it's very clever, and I shall
borrow this.
>>
>> This would also significantly improve the current performance of the
>> getPath() methods on our hierarchical objects because it would be a
>> simple attribute instead of needing to be calculated at runtime. And
>> this would also be much better for implementing equals() on these
>> objects since the true key for a hierarchical object is it's path.
>>
>> The only down side is that the migration path for that would require
>> walking the category and folder trees for all weblogs and saving the
>> path for all of them, which is fairly complex :/
>>
> It seems to me that this is a tradeoff between simplicity and
> performance. Simplicity is storing the local path name and a reference
> to the parent; performance is storing the entire path.
>
> Performance seems to be the better tradeoff here. Operations like
> moving or renaming a path require recursively changing the entire path
> of all children but I assume that the performance benefits outweigh
> the complexity of these infrequent operations.
I'm not sure its simplicity; in theory, since the /pets/dogs is the key
to the sub-category, we can only move/update entries and delete
category/sub categories. Its storage vs performance. Storing the
complete path simplifies the SQL, reduces the query resolution time and
potentially eliminates the need to join against the category table.
>
> Craig
>
>> -- Allen
>>
>>
>>> - Dave
>
> Craig Russell
> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
> 408 276-5638 mailto:Craig.Russell@sun.com
> P.S. A good JDO? O, Gasp!
>
--
* David Levy *
*Principal Engineer*S
Sun Proprietary & Confidential . This e-mail message is for the sole use
of the intended recipient(s) and may contain confidential and
privilidged information. Any unauthorised review, use, disclosure or
distribution is prohibited. If you are not the intended recepient,
please contact the sender by reply e-mail and destroy all copies of the
original message.
Re: Business layer cleanup for 3.2
Posted by Allen Gilliland <al...@sun.com>.
Craig L Russell wrote:
> Hi Allen,
>
> On Nov 14, 2006, at 9:06 AM, Allen Gilliland wrote:
>
>>
>>
>> Dave wrote:
>>> On 11/13/06, Allen Gilliland <al...@sun.com> wrote:
>>>> One of the things that I am planning to do for the 3.2 release is do
>>>> some audit/cleanup of the current business layer code. There are a
>>>> variety of things which could use improving, but the main goal is to
>>>> fix
>>>> our Hibernate configuration so that we are 1) properly using the open
>>>> session in view pattern and 2) enabling lazy fetching on all objects
>>>> and
>>>> associations.
>>> All good.
>>>> Right now our Hibernate config is pretty messy and doesn't take
>>>> advantage of many of Hibernate's performance features, so the main
>>>> reason to do this work is to improve the performance of the business
>>>> layer. The second big reason is just to reduce clutter and simplify
>>>> the
>>>> code as much as possible. There are plenty of places in the code where
>>>> we have methods that aren't used at all or methods which are
>>>> duplicated,
>>>> so those would all be cleaned up.
>>>>
>>>> I have most of this work done already (but not checked in) and there
>>>> aren't really any surprise changes that I had to make except when it
>>>> came to the hierarchical objects. I tried for multiple days to get the
>>>> hierarchical objects to work with the updated hibernate config and the
>>>> current data model, but I kept running into problems. So to fix the
>>>> problem I had to make a small tweak to the way hierarchical objects are
>>>> persisted which fixed my issues and I believe drastically simplifies
>>>> the
>>>> problem overall. The basic change is that I have completely removed
>>>> the
>>>> HierarchicalPersistentObject class and Assoc and it's subclasses and
>>>> changed the data model so that we have a more normal hierarchical
>>>> model.
>>>>
>>>> So, for weblog categories I added a simple 'parentid' column to the
>>>> weblogcategory table and that allows a category to manage relationships
>>>> between it's parent and children directly. Same goes for the
>>>> FolderData
>>>> class, but as it turns out that column already existed in the schema
>>>> but
>>>> wasn't being used. Upgrade path for both of these is fairly simple and
>>>> only requires populating these columns with the right value.
>>> The hierarchical persistent object and the assoc tables were there for
>>> one purpose: to make it possible to do a recursive query (i.e. query
>>> for all entries under /pets and /pets/dogs, /pets/cats etc.) with only
>>> one SQL statement. But that hasn't worked since we made the switch to
>>> the Hibernate Criteria API back in 2004 and we've been doing fine
>>> without it. And the code that started as complex has been moved around
>>> so much that now it's almost unintelligible.
>>> So I'm +1 with that change.
>>
>> Actually, I have an idea that I think can accomplish what's described
>> above and actually kill a couple other birds with the same stone.
>> Basically, if we add another column for the full path of a node like
>> you did above (/pets, /pets/dogs, /pets/cats) then to get all entries
>> under the category /pets by getting all entries that have a category
>> with a path that starts with "/pets". So the sql query would just be ...
>>
>> select * from weblogentry,weblogcategory where weblogentry.categoryid
>> = weblogcategory.id and weblogcategory.path like '/pets%';
>>
>> This would also significantly improve the current performance of the
>> getPath() methods on our hierarchical objects because it would be a
>> simple attribute instead of needing to be calculated at runtime. And
>> this would also be much better for implementing equals() on these
>> objects since the true key for a hierarchical object is it's path.
>>
>> The only down side is that the migration path for that would require
>> walking the category and folder trees for all weblogs and saving the
>> path for all of them, which is fairly complex :/
>>
> It seems to me that this is a tradeoff between simplicity and
> performance. Simplicity is storing the local path name and a reference
> to the parent; performance is storing the entire path.
>
> Performance seems to be the better tradeoff here. Operations like moving
> or renaming a path require recursively changing the entire path of all
> children but I assume that the performance benefits outweigh the
> complexity of these infrequent operations.
I definitely think it would be best to add this new path column and
setup our queries to run like I said above, but I didn't think it needed
to be done yet. I figured we can talk about it and agree on it first,
then look into what the migration path would be and see if we can
accomplish it in a reasonable fashion. My first step was just to
cleanup the actual object model and make it work in a more reliable
fashion, then we can look forward to performance improvements.
If everyone already agrees about adding this new column for the path
then I am happy to start looking at how to implement it as part of 3.2.
-- Allen
>
> Craig
>
>> -- Allen
>>
>>
>>> - Dave
>
> Craig Russell
> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
> 408 276-5638 mailto:Craig.Russell@sun.com
> P.S. A good JDO? O, Gasp!
>
Re: Business layer cleanup for 3.2
Posted by Craig L Russell <Cr...@Sun.COM>.
Hi Allen,
On Nov 14, 2006, at 9:06 AM, Allen Gilliland wrote:
>
>
> Dave wrote:
>> On 11/13/06, Allen Gilliland <al...@sun.com> wrote:
>>> One of the things that I am planning to do for the 3.2 release is do
>>> some audit/cleanup of the current business layer code. There are a
>>> variety of things which could use improving, but the main goal is
>>> to fix
>>> our Hibernate configuration so that we are 1) properly using the
>>> open
>>> session in view pattern and 2) enabling lazy fetching on all
>>> objects and
>>> associations.
>> All good.
>>> Right now our Hibernate config is pretty messy and doesn't take
>>> advantage of many of Hibernate's performance features, so the main
>>> reason to do this work is to improve the performance of the business
>>> layer. The second big reason is just to reduce clutter and
>>> simplify the
>>> code as much as possible. There are plenty of places in the code
>>> where
>>> we have methods that aren't used at all or methods which are
>>> duplicated,
>>> so those would all be cleaned up.
>>>
>>> I have most of this work done already (but not checked in) and there
>>> aren't really any surprise changes that I had to make except when it
>>> came to the hierarchical objects. I tried for multiple days to
>>> get the
>>> hierarchical objects to work with the updated hibernate config
>>> and the
>>> current data model, but I kept running into problems. So to fix the
>>> problem I had to make a small tweak to the way hierarchical
>>> objects are
>>> persisted which fixed my issues and I believe drastically
>>> simplifies the
>>> problem overall. The basic change is that I have completely
>>> removed the
>>> HierarchicalPersistentObject class and Assoc and it's subclasses and
>>> changed the data model so that we have a more normal hierarchical
>>> model.
>>>
>>> So, for weblog categories I added a simple 'parentid' column to the
>>> weblogcategory table and that allows a category to manage
>>> relationships
>>> between it's parent and children directly. Same goes for the
>>> FolderData
>>> class, but as it turns out that column already existed in the
>>> schema but
>>> wasn't being used. Upgrade path for both of these is fairly
>>> simple and
>>> only requires populating these columns with the right value.
>> The hierarchical persistent object and the assoc tables were there
>> for
>> one purpose: to make it possible to do a recursive query (i.e. query
>> for all entries under /pets and /pets/dogs, /pets/cats etc.) with
>> only
>> one SQL statement. But that hasn't worked since we made the switch to
>> the Hibernate Criteria API back in 2004 and we've been doing fine
>> without it. And the code that started as complex has been moved
>> around
>> so much that now it's almost unintelligible.
>> So I'm +1 with that change.
>
> Actually, I have an idea that I think can accomplish what's
> described above and actually kill a couple other birds with the
> same stone. Basically, if we add another column for the full path
> of a node like you did above (/pets, /pets/dogs, /pets/cats) then
> to get all entries under the category /pets by getting all entries
> that have a category with a path that starts with "/pets". So the
> sql query would just be ...
>
> select * from weblogentry,weblogcategory where
> weblogentry.categoryid = weblogcategory.id and weblogcategory.path
> like '/pets%';
>
> This would also significantly improve the current performance of
> the getPath() methods on our hierarchical objects because it would
> be a simple attribute instead of needing to be calculated at
> runtime. And this would also be much better for implementing equals
> () on these objects since the true key for a hierarchical object is
> it's path.
>
> The only down side is that the migration path for that would
> require walking the category and folder trees for all weblogs and
> saving the path for all of them, which is fairly complex :/
>
It seems to me that this is a tradeoff between simplicity and
performance. Simplicity is storing the local path name and a
reference to the parent; performance is storing the entire path.
Performance seems to be the better tradeoff here. Operations like
moving or renaming a path require recursively changing the entire
path of all children but I assume that the performance benefits
outweigh the complexity of these infrequent operations.
Craig
> -- Allen
>
>
>> - Dave
Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
408 276-5638 mailto:Craig.Russell@sun.com
P.S. A good JDO? O, Gasp!
Re: Business layer cleanup for 3.2
Posted by Allen Gilliland <al...@sun.com>.
Dave wrote:
> On 11/13/06, Allen Gilliland <al...@sun.com> wrote:
>> One of the things that I am planning to do for the 3.2 release is do
>> some audit/cleanup of the current business layer code. There are a
>> variety of things which could use improving, but the main goal is to fix
>> our Hibernate configuration so that we are 1) properly using the open
>> session in view pattern and 2) enabling lazy fetching on all objects and
>> associations.
>
> All good.
>
>> Right now our Hibernate config is pretty messy and doesn't take
>> advantage of many of Hibernate's performance features, so the main
>> reason to do this work is to improve the performance of the business
>> layer. The second big reason is just to reduce clutter and simplify the
>> code as much as possible. There are plenty of places in the code where
>> we have methods that aren't used at all or methods which are duplicated,
>> so those would all be cleaned up.
>>
>> I have most of this work done already (but not checked in) and there
>> aren't really any surprise changes that I had to make except when it
>> came to the hierarchical objects. I tried for multiple days to get the
>> hierarchical objects to work with the updated hibernate config and the
>> current data model, but I kept running into problems. So to fix the
>> problem I had to make a small tweak to the way hierarchical objects are
>> persisted which fixed my issues and I believe drastically simplifies the
>> problem overall. The basic change is that I have completely removed the
>> HierarchicalPersistentObject class and Assoc and it's subclasses and
>> changed the data model so that we have a more normal hierarchical model.
>>
>> So, for weblog categories I added a simple 'parentid' column to the
>> weblogcategory table and that allows a category to manage relationships
>> between it's parent and children directly. Same goes for the FolderData
>> class, but as it turns out that column already existed in the schema but
>> wasn't being used. Upgrade path for both of these is fairly simple and
>> only requires populating these columns with the right value.
>
> The hierarchical persistent object and the assoc tables were there for
> one purpose: to make it possible to do a recursive query (i.e. query
> for all entries under /pets and /pets/dogs, /pets/cats etc.) with only
> one SQL statement. But that hasn't worked since we made the switch to
> the Hibernate Criteria API back in 2004 and we've been doing fine
> without it. And the code that started as complex has been moved around
> so much that now it's almost unintelligible.
>
> So I'm +1 with that change.
Actually, I have an idea that I think can accomplish what's described
above and actually kill a couple other birds with the same stone.
Basically, if we add another column for the full path of a node like you
did above (/pets, /pets/dogs, /pets/cats) then to get all entries under
the category /pets by getting all entries that have a category with a
path that starts with "/pets". So the sql query would just be ...
select * from weblogentry,weblogcategory where weblogentry.categoryid =
weblogcategory.id and weblogcategory.path like '/pets%';
This would also significantly improve the current performance of the
getPath() methods on our hierarchical objects because it would be a
simple attribute instead of needing to be calculated at runtime. And
this would also be much better for implementing equals() on these
objects since the true key for a hierarchical object is it's path.
The only down side is that the migration path for that would require
walking the category and folder trees for all weblogs and saving the
path for all of them, which is fairly complex :/
-- Allen
>
> - Dave
Re: Business layer cleanup for 3.2
Posted by Dave <sn...@gmail.com>.
On 11/13/06, Allen Gilliland <al...@sun.com> wrote:
> One of the things that I am planning to do for the 3.2 release is do
> some audit/cleanup of the current business layer code. There are a
> variety of things which could use improving, but the main goal is to fix
> our Hibernate configuration so that we are 1) properly using the open
> session in view pattern and 2) enabling lazy fetching on all objects and
> associations.
All good.
> Right now our Hibernate config is pretty messy and doesn't take
> advantage of many of Hibernate's performance features, so the main
> reason to do this work is to improve the performance of the business
> layer. The second big reason is just to reduce clutter and simplify the
> code as much as possible. There are plenty of places in the code where
> we have methods that aren't used at all or methods which are duplicated,
> so those would all be cleaned up.
>
> I have most of this work done already (but not checked in) and there
> aren't really any surprise changes that I had to make except when it
> came to the hierarchical objects. I tried for multiple days to get the
> hierarchical objects to work with the updated hibernate config and the
> current data model, but I kept running into problems. So to fix the
> problem I had to make a small tweak to the way hierarchical objects are
> persisted which fixed my issues and I believe drastically simplifies the
> problem overall. The basic change is that I have completely removed the
> HierarchicalPersistentObject class and Assoc and it's subclasses and
> changed the data model so that we have a more normal hierarchical model.
>
> So, for weblog categories I added a simple 'parentid' column to the
> weblogcategory table and that allows a category to manage relationships
> between it's parent and children directly. Same goes for the FolderData
> class, but as it turns out that column already existed in the schema but
> wasn't being used. Upgrade path for both of these is fairly simple and
> only requires populating these columns with the right value.
The hierarchical persistent object and the assoc tables were there for
one purpose: to make it possible to do a recursive query (i.e. query
for all entries under /pets and /pets/dogs, /pets/cats etc.) with only
one SQL statement. But that hasn't worked since we made the switch to
the Hibernate Criteria API back in 2004 and we've been doing fine
without it. And the code that started as complex has been moved around
so much that now it's almost unintelligible.
So I'm +1 with that change.
- Dave