You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@isis.apache.org by Barry Woods <ba...@bigpond.com> on 2011/08/24 18:15:47 UTC
Infinite recursion with JDBC SQL Object store
Hi Dan,
Have been following the evolution of NO into ISIS with interest. We are
using ISIS to develop a major application and we have been using the XML
persistor for most development undertaken to date. This project includes the
development of a custom ISIS viewer based on GWT for a specialized UI that
is now in beta.
We recently initiated a move to the SQL object store so we can get
multi-user operating. We have implemented this using Oracle XE DB.
We now have an issue with infinite recursion occuring in the
AbtractAutoMapper.setupFullMapping method for a domain class that implements
a hierarchy i.e. it contains a collection of the same class. It ultimately
crashes with stack overflow. I have attached a log extract.
This same structure works fine using the XML persistor. This issue has
brought our SQL implementation to a halt as the class involved is a core
class in our model (murphy's law again !)
I really don't understand this part of the framework sufficiently to debug
this. Any pointers or assistance you can provide would be greatly
appreciated.
Regards,
Barry Woods
P.S. Not sure whether this is the right place to raise issues - pls advise
if not.
Re: Infinite recursion with JDBC SQL Object store
Posted by Kevin Meyer - KMZ <ke...@kmz.co.za>.
Hi Barry,
> P.S. Notsure whether this is the right place to raise issues - pls
> advise if not.
This is probably the best place... welcome!
On 25 Aug 2011 at 2:15, Barry Woods wrote:
> We recently initiated amove to the SQL object store so we can get
> multi-user operating. We have implemented this using Oracle XE DB.
>
> We now have an issue with infinite recursionoccuring in the
> AbtractAutoMapper.setupFullMapping method for a domain class that
> implements a hierarchy i.e. it contains a collection ofthesame
> class. It ultimately crashes with stack overflow. I have attached a
> log extract.
The initial SQL Objectstore (SQL OS) development was done (I think)
by Robert Matthews. I more or less took over from him and have been
extending it for the last year or so.
Having said this - a lot of the SQL OS is a bit black-box to me - I've
worked on the bits that I've wanted to extend, and taken for granted
that the rest works!
I can fully believe that what you describe happens as you describe it -
I'm not aware of there being any logic in place to halt the automatic
introspection.
> This same structureworks fine using the XML persistor. This issue
> has brought our SQL implementationto a halt as the class involved is
> a core class in our model (murphy's law again !)
Hmm. I have to rely on input from Dan or Robert to understand what is
happening in the XML OS vs the SQL OS.
> I really don't understand this part of the frameworksufficiently to
> debug this. Any pointers or assistance you can provide would be
> greatly appreciated.
Let me sleep on this. I should have some time tomorrow night (and
following) to try and reproduce your problem and see what I can come
up with.
My immediate suggestion is to consider a bit of jiggery: can you make
your class implement an (empty) interface, and have your collection be
a collection of the interface instead of the full class? This ought to
prevent the infinite recursion, but I'm not sure what else it'll break.
In the mean time, maybe someone else (Rob? Dan?) can provide
some hints.
Regards,
Kevin
Re: Infinite recursion with JDBC SQL Object store
Posted by Kevin Meyer - KMZ <ke...@kmz.co.za>.
Hi Barry,
I've looked a little into this problem, and I'm getting the impression that
the best approach would be to use the interface method I first
described - just ensure that your interface defines at least 1 property
that satisfies the object store (i.e. choose a getter that your Node class
implements). This will stop the "no properties" exception.
It will not work yet, as the MultiColumnCombinedCollectionMapper is
only picked up on the interface, not the subclasses (the same is true
for any class that extends another - only the super class is handled,
not the sub-classes).
I need to fix that, too - see below.
Basically, I can't yet figure out how to stop the recursion problem.
I have also confirmed that the polymorphic collection is not re-loaded
(using a colection of the "BaseClass " type, with instances of SubOne,
SubTwo, as defined previously).
viz:
class BaseClass {
}
class SubOne extends BaseClass {
}
class SubTwo extends BaseClass {
}
I don't know how difficult this will be to fix - I need to add all properties
of super classes/interfaces to a polymorphic class...
Regards,
Kevin
On 1 Sep 2011 at 13:07, Barry Woods wrote:
> Hi Dan & Kevin,
>
> Thank you both for your prompt replies and apologies for my late response
> but I have been dragged on to other work the last several days.
>
> Since sending my original email I have had a closer look in an effort to
> better understand the JDBC object store codebase. It occurs to me that a
> possible fix is for changes to the AbtractAutoMapper.setupFullMapping method
> to ignore further mapping of any association fields of the same type as the
> class being mapped. This would fix the recursion issue but I am unsure
> whether that would have any undesirable side effects.
>
> I will raise a JIRA for this issue later today and attach the log extract.
>
> > My immediate suggestion is to consider a bit of jiggery: can you make your
> class implement an (empty) interface, and > have your collection be a
> collection of the interface instead of the full class? This ought to prevent
> the infinite > > recursion, but I'm not sure what else it'll break.
>
> Kevin, unfortunately your suggestion won't work because the JDBC persistor
> doesn't support polymorphism. This is a fundamental problem for us as our
> domain model depends heavily on polymorphic associations.
>
> Our domain models a directed graph of nodes and the relations between them.
> We have a base class named (not surprisingly!) Node. These nodes can be
> represented by one of several built-in subtypes, but in addition the model
> supports dynamic extension allowing for new node types and hence subclasses
> to be added. The relationships between nodes are represented by LinkObject
> instances.
>
> In order for dynamic extensions to work it is essential that the LinkObject
> defines the from/to node associations as the shared supertype i.e. as Node
> because any node can have a relationship to any other node (subject to some
> business rules). This strategy works fine for the XML persistor but falls
> down with the JDBC persistor.
>
> Examination of this problem shows that when nodes are added to the graph
> everything works nicely during the same persistence session. The problem
> arises when a new persistence session is started (e.g. when starting the
> application again) and the framework commences re-instantiation and
> rehydration of the persisted objects. There is no information in the
> persisted LinkObject rows about the original subtypes and so it attempts to
> re-instantiate the objects as supertype instances which eventually falls
> over. Presumably the XML persistor works because the stored association
> fields are type agnostic.
>
> It occurred to me that I could use the JDO strategy for dealing with
> polymorphism which is to store the persisted primary key as a primitive
> value rather than as a reference in the LinkObject association. We could
> also store a fully qualified class name for the from/to associations
> providing all the information required for re-instantiation. This of course
> presents some difficulties with ISIS because the OID is not accessible from
> the applib. Presumably even if the OID was available, to make this useful we
> would also need an additional query method to query by OID value.
>
> I would be very interested in your views on the suggestion above and any
> other strategies you might suggest for overcoming theses issues.
>
> >> using ISIS to develop a major application and we have been using the
> >> XML persistor for most development undertaken to date. This project
> >> includes the development of a custom ISIS viewer based on GWT for a
> >> specialized UI that is now in beta.
>
> > That's a pretty major undertaking; it'd be nice to hear more about this.
>
> Dan, the custom UI presents tailored views for each of the nodes according
> to type. In addition it will allow us to view the graph as a graph and to
> zoom in to different parts of the graph to see immediate relations. Clicking
> on a node displayed in the graph will open the tailored view. If you
> interested I will provide more details as the design of this part of the
> viewer is bedded down.
>
> Regards,
>
> Barry
>
> -----Original Message-----
> From: Dan Haywood [mailto:dkhaywood@gmail.com] On Behalf Of Dan Haywood
> Sent: Friday, 26 August 2011 4:45 AM
> To: isis-dev@incubator.apache.org
> Subject: Re: Infinite recursion with JDBC SQL Object store
>
> On 24/08/2011 17:15, Barry Woods wrote:
> > Hi Dan,
> > Have been following the evolution of NO into ISIS with interest.
> Good to hear!
>
>
> We are
> > using ISIS to develop a major application and we have been using the
> > XML persistor for most development undertaken to date. This project
> > includes the development of a custom ISIS viewer based on GWT for a
> > specialized UI that is now in beta.
>
> That's a pretty major undertaking; it'd be nice to hear more about this.
>
>
> > We recently initiated a move to the SQL object store so we can get
> > multi-user operating...I really don't understand this part of the
> > framework sufficiently to debug this. Any pointers or assistance you
> > can provide would be greatly appreciated.
>
> I'm afraid I can't help you much, at least insofar as Rob wrote this code
> originally and I've not had the need to look into it. I see from Kevin's
> reply that he's going to see if he can assist. If he gets nowhere then I'll
> see what I can do.
>
> The other thing that I am happy to do - which might be another way forward
> for you - is to resurrect the JPA object store that I wrote against NOF.
> This isn't part of Isis because it uses Hibernate, which is LGPL (not
> compatible with ASF policy). However, on my todo list is to update the code
> as it currently is and to check into apache-extras.org [1] (a subsequent
> todo is to port it from Hibernate to OpenJPA so it can become part of Isis
> proper). At any rate, there's coverage of the JPA object store in my book
> [2], so you can evaluate if it might be a fit for you.
>
> Sorry not to have an immediate fix for you. Could you also raise a JIRA for
> us, though, and attach that log to it?
>
> Thx
> Dan
>
>
> > Regards,
> > Barry Woods
> > P.S. Not sure whether this is the right place to raise issues - pls
> > advise if not.
> Indeed it is!
>
> [1] http://code.google.com/a/apache-extras.org/
> [2] http://pragprog.com/titles/dhnako
>
--
Kevin Meyer, PhD, Pr.Sci.Nat
KMZ P.O. Box 9822, Sharon Park, South Africa.
Tel: +27 11 363 2001 Cell: +27 83 346 3045
Re: Infinite recursion with JDBC SQL Object store
Posted by Kevin Meyer - KMZ <ke...@kmz.co.za>.
Hi Barry,
> Thank you both for your prompt replies and apologies for my late response
> but I have been dragged on to other work the last several days.
Great that you came back, though. I've had a rough week myself - and
have not had a chance to look into this. Things are looking better next
week, though...
> Since sending my original email I have had a closer look in an effort to
> better understand the JDBC object store codebase. It occurs to me that a
> possible fix is for changes to the AbtractAutoMapper.setupFullMapping method
> to ignore further mapping of any association fields of the same type as the
> class being mapped. This would fix the recursion issue but I am unsure
> whether that would have any undesirable side effects.
Thanks - I'll see how this can be done. I'll also try and add some tests
to examine this behaviour.
> > My immediate suggestion is to consider a bit of jiggery: can you make your
> class implement an (empty) interface, and > have your collection be a
> collection of the interface instead of the full class? This ought to prevent
> the infinite > > recursion, but I'm not sure what else it'll break.
>
> Kevin, unfortunately your suggestion won't work because the JDBC persistor
> doesn't support polymorphism. This is a fundamental problem for us as our
> domain model depends heavily on polymorphic associations.
I can't remember testing this - you may be right.
The one behaviour I *have* noticed is that the SQL-OS does not
correctly return all instances of sub-classes. Consider:
class BaseClass {
}
class SubOne extends BaseClass {
}
class SubTwo extends BaseClass {
}
then the Factory/Repository method:
allInstances(BaseClass.class) does not return instances of SubOne
and SubTwo - you'd have to manually fetch instances of them and add
them to the list.
I seem to recall that the XML Persistor worked as expected.
> In order for dynamic extensions to work it is essential that the LinkObject
> defines the from/to node associations as the shared supertype i.e. as Node
> because any node can have a relationship to any other node (subject to some
> business rules). This strategy works fine for the XML persistor but falls
> down with the JDBC persistor.
Is this related to what I just listed, above?
> Examination of this problem shows that when nodes are added to the graph
> everything works nicely during the same persistence session. The problem
> arises when a new persistence session is started (e.g. when starting the
> application again) and the framework commences re-instantiation and
> rehydration of the persisted objects. There is no information in the
> persisted LinkObject rows about the original subtypes and so it attempts to
> re-instantiate the objects as supertype instances which eventually falls
> over. Presumably the XML persistor works because the stored association
> fields are type agnostic.
Again, I need to write some test code to demonstrate this behaviour.
If I can repeat: you're saying that, using my example classes above, if:
class Node {
List<BaseClass> baseClasses(){}
}
then the baseClasses collection is not correctly restored with instances
of SubOne and SubTwo, as expected?
> It occurred to me that I could use the JDO strategy for dealing with
> polymorphism which is to store the persisted primary key as a primitive
> value rather than as a reference in the LinkObject association. We could
> also store a fully qualified class name for the from/to associations
> providing all the information required for re-instantiation. This of course
> presents some difficulties with ISIS because the OID is not accessible from
> the applib. Presumably even if the OID was available, to make this useful we
> would also need an additional query method to query by OID value.
To support this, your domain repository interface should include
methods of the form:
interface NodeFactory {
Oid oidFor(Node node);
Node nodeFor(Oid oid);
}
then we just need to interact with the actual persistor to map between
the (SqlOid or SerialOid? What does the XML Persistor use?) and the
object.
Dan? What's your feeling / input here?
> I would be very interested in your views on the suggestion above and any
> other strategies you might suggest for overcoming theses issues.
How does Hibernate handle your situation?
> Regards,
>
> Barry
>
Regards,
Kevin
RE: Infinite recursion with JDBC SQL Object store
Posted by Barry Woods <ba...@bigpond.com>.
Hi Dan & Kevin,
Thank you both for your prompt replies and apologies for my late response
but I have been dragged on to other work the last several days.
Since sending my original email I have had a closer look in an effort to
better understand the JDBC object store codebase. It occurs to me that a
possible fix is for changes to the AbtractAutoMapper.setupFullMapping method
to ignore further mapping of any association fields of the same type as the
class being mapped. This would fix the recursion issue but I am unsure
whether that would have any undesirable side effects.
I will raise a JIRA for this issue later today and attach the log extract.
> My immediate suggestion is to consider a bit of jiggery: can you make your
class implement an (empty) interface, and > have your collection be a
collection of the interface instead of the full class? This ought to prevent
the infinite > > recursion, but I'm not sure what else it'll break.
Kevin, unfortunately your suggestion won't work because the JDBC persistor
doesn't support polymorphism. This is a fundamental problem for us as our
domain model depends heavily on polymorphic associations.
Our domain models a directed graph of nodes and the relations between them.
We have a base class named (not surprisingly!) Node. These nodes can be
represented by one of several built-in subtypes, but in addition the model
supports dynamic extension allowing for new node types and hence subclasses
to be added. The relationships between nodes are represented by LinkObject
instances.
In order for dynamic extensions to work it is essential that the LinkObject
defines the from/to node associations as the shared supertype i.e. as Node
because any node can have a relationship to any other node (subject to some
business rules). This strategy works fine for the XML persistor but falls
down with the JDBC persistor.
Examination of this problem shows that when nodes are added to the graph
everything works nicely during the same persistence session. The problem
arises when a new persistence session is started (e.g. when starting the
application again) and the framework commences re-instantiation and
rehydration of the persisted objects. There is no information in the
persisted LinkObject rows about the original subtypes and so it attempts to
re-instantiate the objects as supertype instances which eventually falls
over. Presumably the XML persistor works because the stored association
fields are type agnostic.
It occurred to me that I could use the JDO strategy for dealing with
polymorphism which is to store the persisted primary key as a primitive
value rather than as a reference in the LinkObject association. We could
also store a fully qualified class name for the from/to associations
providing all the information required for re-instantiation. This of course
presents some difficulties with ISIS because the OID is not accessible from
the applib. Presumably even if the OID was available, to make this useful we
would also need an additional query method to query by OID value.
I would be very interested in your views on the suggestion above and any
other strategies you might suggest for overcoming theses issues.
>> using ISIS to develop a major application and we have been using the
>> XML persistor for most development undertaken to date. This project
>> includes the development of a custom ISIS viewer based on GWT for a
>> specialized UI that is now in beta.
> That's a pretty major undertaking; it'd be nice to hear more about this.
Dan, the custom UI presents tailored views for each of the nodes according
to type. In addition it will allow us to view the graph as a graph and to
zoom in to different parts of the graph to see immediate relations. Clicking
on a node displayed in the graph will open the tailored view. If you
interested I will provide more details as the design of this part of the
viewer is bedded down.
Regards,
Barry
-----Original Message-----
From: Dan Haywood [mailto:dkhaywood@gmail.com] On Behalf Of Dan Haywood
Sent: Friday, 26 August 2011 4:45 AM
To: isis-dev@incubator.apache.org
Subject: Re: Infinite recursion with JDBC SQL Object store
On 24/08/2011 17:15, Barry Woods wrote:
> Hi Dan,
> Have been following the evolution of NO into ISIS with interest.
Good to hear!
We are
> using ISIS to develop a major application and we have been using the
> XML persistor for most development undertaken to date. This project
> includes the development of a custom ISIS viewer based on GWT for a
> specialized UI that is now in beta.
That's a pretty major undertaking; it'd be nice to hear more about this.
> We recently initiated a move to the SQL object store so we can get
> multi-user operating...I really don't understand this part of the
> framework sufficiently to debug this. Any pointers or assistance you
> can provide would be greatly appreciated.
I'm afraid I can't help you much, at least insofar as Rob wrote this code
originally and I've not had the need to look into it. I see from Kevin's
reply that he's going to see if he can assist. If he gets nowhere then I'll
see what I can do.
The other thing that I am happy to do - which might be another way forward
for you - is to resurrect the JPA object store that I wrote against NOF.
This isn't part of Isis because it uses Hibernate, which is LGPL (not
compatible with ASF policy). However, on my todo list is to update the code
as it currently is and to check into apache-extras.org [1] (a subsequent
todo is to port it from Hibernate to OpenJPA so it can become part of Isis
proper). At any rate, there's coverage of the JPA object store in my book
[2], so you can evaluate if it might be a fit for you.
Sorry not to have an immediate fix for you. Could you also raise a JIRA for
us, though, and attach that log to it?
Thx
Dan
> Regards,
> Barry Woods
> P.S. Not sure whether this is the right place to raise issues - pls
> advise if not.
Indeed it is!
[1] http://code.google.com/a/apache-extras.org/
[2] http://pragprog.com/titles/dhnako
Re: Infinite recursion with JDBC SQL Object store
Posted by Dan Haywood <da...@haywood-associates.co.uk>.
On 24/08/2011 17:15, Barry Woods wrote:
> Hi Dan,
> Have been following the evolution of NO into ISIS with interest.
Good to hear!
We are
> using ISIS to develop a major application and we have been using the XML
> persistor for most development undertaken to date. This project includes
> the development of a custom ISIS viewer based on GWT for a specialized
> UI that is now in beta.
That's a pretty major undertaking; it'd be nice to hear more about this.
> We recently initiated a move to the SQL object store so we can get
> multi-user operating...I really don't understand this part of the framework sufficiently to
> debug this. Any pointers or assistance you can provide would be greatly
> appreciated.
I'm afraid I can't help you much, at least insofar as Rob wrote this
code originally and I've not had the need to look into it. I see from
Kevin's reply that he's going to see if he can assist. If he gets
nowhere then I'll see what I can do.
The other thing that I am happy to do - which might be another way
forward for you - is to resurrect the JPA object store that I wrote
against NOF. This isn't part of Isis because it uses Hibernate, which
is LGPL (not compatible with ASF policy). However, on my todo list is
to update the code as it currently is and to check into
apache-extras.org [1] (a subsequent todo is to port it from Hibernate to
OpenJPA so it can become part of Isis proper). At any rate, there's
coverage of the JPA object store in my book [2], so you can evaluate if
it might be a fit for you.
Sorry not to have an immediate fix for you. Could you also raise a JIRA
for us, though, and attach that log to it?
Thx
Dan
> Regards,
> Barry Woods
> P.S. Not sure whether this is the right place to raise issues - pls
> advise if not.
Indeed it is!
[1] http://code.google.com/a/apache-extras.org/
[2] http://pragprog.com/titles/dhnako