You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cayenne.apache.org by Kevin Menard <km...@servprise.com> on 2007/10/16 00:25:40 UTC
Reconciling DataContexts
So, if there is one thing that drives me nuts about Cayenne, it's managing
ObjectContexts. In particular, you cannot relate two Persistent objects
without first putting them into an ObjectContext. If one is committed and
the other is not, you can have them in different contexts, but for newly
created objects, this is a major pain in the neck.
Since I've been complaining about it for probably close to three years now,
I'd like to finally do something about it.
Here are my rough notes from the airport:
OK Cases:
- Objects in same context
- Objects in different contexts, but objects are committed already
Don't work, but should:
- Objects in null contexts
- Objects in different contexts, but same data maps and domains
Very hard to say, probably okay if don't work:
- Objects in different contexts, contexts have different data maps
- Objects in different contexts, contexts have different data domains
As I started actually digging into the code, I ran into a lot of NPE issues
trying to associate two Persistent objects with no context with one another.
In an attempt to prevent adding special null-logic handling, I thought about
applying the Null Object pattern to the problem. The basic idea is rather
than use null as the default objectContext for CDO, use an instance of
DelayedContext. The problem here is having objects in different contexts.
So, it appears by fixing one, you can essentially fix the other.
To address the latter, I was looking to have a Set of ObjectContexts stored
in either BaseContext or DataContext. When willConnect() is called, you'd
have something like the following:
else if (this.getObjectContext().getEntityResolver() ==
object.getObjectContext().getEntityResolver()) {
((DataContext)
this.getObjectContext()).addContextToMergeWith(object.getObjectContext());
((DataContext)
object.getObjectContext()).addContextToMergeWith(this.getObjectContext());
}
else {
throw new CayenneRuntimeException(
"Cannot set object as destination of relationship "
+ relationshipName
+ " because it is in a different DataMap or
DataDomain.");
}
(Casts are just an artifact of me screwing around).
What I'm thinking would happen is that when commitChanges() is called, the
set of contexts to be merged with will be iterated and any changes applied
to the current object store / object store graph diff. The relationship is
bidirectional so that the user can initiate the commit from any object
registered with any context.
Here is about where I lose it. I'm not as well-versed in the internal
going-ons of Cayenne as I would like to be. It appears Cayenne goes to
great efforts to essentially cache the graph manipulations so as to avoid a
full traversal. I really don't know, though.
Caching ordering of operations could make this tricky, but in principal
should be wholly doable.
If anyone has any thoughts on this or can fill in any missing pieces, I'd
appreciate it. This is really something I'd like to see fixed sooner rather
than later. I think it may be a requisite for JPA compliance as well.
--
Kevin
Re: Reconciling DataContexts
Posted by Kevin Menard <km...@servprise.com>.
On 10/16/07 9:18 AM, "Michael Gentry" <bl...@gmail.com> wrote:
> Kevin,
>
> Could a nested context help with your situation?
>
> /dev/mrg
Unless I've missed how they work, not entirely. I've used them, but the
difference between them and peer data contexts is fairly minimal. You still
have to use localObject().
--
Kevin
Re: Reconciling DataContexts
Posted by Michael Gentry <bl...@gmail.com>.
Kevin,
Could a nested context help with your situation?
/dev/mrg
On 10/16/07, Kevin Menard <km...@servprise.com> wrote:
> The problem with localObject is that it does not traverse the whole graph,
> making it practically useless in all but the most trivial situations.
> Manually copying over the graph is doable, but a lot of work for something
> that really should be far easier.
>
> For example, we have customers and orders. All orders are associated with
> customers. We keep the customer info and order info in two different
> contexts for the most part, because we don't want to prematurely commit an
> order while a customer may feel it necessary to change his shipping address,
> for example.
>
> What we've resorted to doing is committing the customer data, then doing a
> requery based on the ObjectID using the order context. This allows us to
> get a fully populated object. Of course, this means more trips to the DB
> and a lot of committing/refetching code. It works, but it hardly makes
> Cayenne transparent.
>
> The whole thing becomes fragile, and there's a lot that Cayenne could do to
> help out. I suppose the problem is exacerbated by the null context problem.
> If we fix that, we'd likely be in much better shape. As it stands, with our
> Tapestry app, we have to stage the objects at page render time to allow the
> association of different objects. That creates for a lot of headaches as
> well, and a lot of context management that really isn't necessary.
>
> --
> Kevin
>
>
> On 10/16/07 9:02 AM, "Andrus Adamchik" <an...@objectstyle.org> wrote:
>
> > Hi Kevin,
> >
> > Relating (as in "creating a direct reference in a Java sense") two
> > objects belonging to two distinct contexts breaks fundamental Cayenne
> > design, most notably assumptions about uniquing and transaction
> > boundaries. I am -1 on that until somebody persuades me that this is
> > a good idea and explains how to avoid messing up existing
> > assumptions. (FWIW there is a workaround - referencing a peer of a
> > given object in another context via 'localObject').
> >
> >
> > On the other hand the inability to relate two TRANSIENT objects (i.e.
> > objects without a context) is indeed a shortcuming. Here is one way
> > how it might work (following the JPA patterns) :
> >
> > When such relationship is established, we would not attempt to create
> > a reverse relationship (something that would require an
> > EntityResolver to be present). We just create it one-way. So a user
> > can build an object graph of an arbitrary size without a context and
> > then at some point do one of the following with it:
> >
> > * "ObjectContext.registerNewObject(..)" (existing; equivalent to JPA
> > "persist" method)
> > * "ObjectContext.aNewMethod(..)" (does not exist yet; equivalent to
> > JPA "merge" method and somewhat of an equivalent of a "localObject"
> > method).
> >
> > Both would traverse a graph of transient objects (since they are not
> > persistent yet, the graph is assumed to be finite and will not
> > trigger sucking the entire DB in memory), attach them to the context
> > and connect missing reverse relationships. The difference is that in
> > the first case we'd assume that the objects are not yet persistent,
> > while in a second case we'd attempt to match them against existing DB
> > rows. (a typical use of a second case would be receiving XML-
> > serialized stream of objects corresponding to the existing data).
> >
> >
> > But you have something different in mind? Could you elaborate on the
> > use cases?
> >
> > Thanks
> > Andrus
> >
> >
> > On Oct 16, 2007, at 1:25 AM, Kevin Menard wrote:
> >> So, if there is one thing that drives me nuts about Cayenne, it's
> >> managing
> >> ObjectContexts. In particular, you cannot relate two Persistent
> >> objects
> >> without first putting them into an ObjectContext. If one is
> >> committed and
> >> the other is not, you can have them in different contexts, but for
> >> newly
> >> created objects, this is a major pain in the neck.
> >>
> >> Since I've been complaining about it for probably close to three
> >> years now,
> >> I'd like to finally do something about it.
> >>
> >> Here are my rough notes from the airport:
> >>
> >> OK Cases:
> >>
> >> - Objects in same context
> >> - Objects in different contexts, but objects are committed already
> >>
> >> Don't work, but should:
> >>
> >> - Objects in null contexts
> >> - Objects in different contexts, but same data maps and domains
> >>
> >> Very hard to say, probably okay if don't work:
> >>
> >> - Objects in different contexts, contexts have different data maps
> >> - Objects in different contexts, contexts have different data domains
> >>
> >>
> >> As I started actually digging into the code, I ran into a lot of
> >> NPE issues
> >> trying to associate two Persistent objects with no context with one
> >> another.
> >> In an attempt to prevent adding special null-logic handling, I
> >> thought about
> >> applying the Null Object pattern to the problem. The basic idea is
> >> rather
> >> than use null as the default objectContext for CDO, use an instance of
> >> DelayedContext. The problem here is having objects in different
> >> contexts.
> >> So, it appears by fixing one, you can essentially fix the other.
> >>
> >> To address the latter, I was looking to have a Set of
> >> ObjectContexts stored
> >> in either BaseContext or DataContext. When willConnect() is
> >> called, you'd
> >> have something like the following:
> >>
> >> else if (this.getObjectContext().getEntityResolver() ==
> >> object.getObjectContext().getEntityResolver()) {
> >> ((DataContext)
> >> this.getObjectContext()).addContextToMergeWith
> >> (object.getObjectContext());
> >> ((DataContext)
> >> object.getObjectContext()).addContextToMergeWith
> >> (this.getObjectContext());
> >> }
> >> else {
> >> throw new CayenneRuntimeException(
> >> "Cannot set object as destination of
> >> relationship "
> >> + relationshipName
> >> + " because it is in a different
> >> DataMap or
> >> DataDomain.");
> >> }
> >>
> >> (Casts are just an artifact of me screwing around).
> >>
> >> What I'm thinking would happen is that when commitChanges() is
> >> called, the
> >> set of contexts to be merged with will be iterated and any changes
> >> applied
> >> to the current object store / object store graph diff. The
> >> relationship is
> >> bidirectional so that the user can initiate the commit from any object
> >> registered with any context.
> >>
> >> Here is about where I lose it. I'm not as well-versed in the internal
> >> going-ons of Cayenne as I would like to be. It appears Cayenne
> >> goes to
> >> great efforts to essentially cache the graph manipulations so as to
> >> avoid a
> >> full traversal. I really don't know, though.
> >>
> >> Caching ordering of operations could make this tricky, but in
> >> principal
> >> should be wholly doable.
> >>
> >> If anyone has any thoughts on this or can fill in any missing
> >> pieces, I'd
> >> appreciate it. This is really something I'd like to see fixed
> >> sooner rather
> >> than later. I think it may be a requisite for JPA compliance as well.
> >>
> >> --
> >> Kevin
> >>
> >>
> >
>
> --
>
>
>
Re: Reconciling DataContexts
Posted by Andrus Adamchik <an...@objectstyle.org>.
BTW, note that JPA "merge" returns a merged object, so it is exactly
like "localObject" in that it locates a copy of an object local to
the context.
Andrus
On Oct 16, 2007, at 10:02 AM, Andrus Adamchik wrote:
> Hi Kevin,
>
> Relating (as in "creating a direct reference in a Java sense") two
> objects belonging to two distinct contexts breaks fundamental
> Cayenne design, most notably assumptions about uniquing and
> transaction boundaries. I am -1 on that until somebody persuades me
> that this is a good idea and explains how to avoid messing up
> existing assumptions. (FWIW there is a workaround - referencing a
> peer of a given object in another context via 'localObject').
>
>
> On the other hand the inability to relate two TRANSIENT objects
> (i.e. objects without a context) is indeed a shortcuming. Here is
> one way how it might work (following the JPA patterns) :
>
> When such relationship is established, we would not attempt to
> create a reverse relationship (something that would require an
> EntityResolver to be present). We just create it one-way. So a user
> can build an object graph of an arbitrary size without a context
> and then at some point do one of the following with it:
>
> * "ObjectContext.registerNewObject(..)" (existing; equivalent to
> JPA "persist" method)
> * "ObjectContext.aNewMethod(..)" (does not exist yet; equivalent to
> JPA "merge" method and somewhat of an equivalent of a "localObject"
> method).
>
> Both would traverse a graph of transient objects (since they are
> not persistent yet, the graph is assumed to be finite and will not
> trigger sucking the entire DB in memory), attach them to the
> context and connect missing reverse relationships. The difference
> is that in the first case we'd assume that the objects are not yet
> persistent, while in a second case we'd attempt to match them
> against existing DB rows. (a typical use of a second case would be
> receiving XML-serialized stream of objects corresponding to the
> existing data).
>
>
> But you have something different in mind? Could you elaborate on
> the use cases?
>
> Thanks
> Andrus
>
>
> On Oct 16, 2007, at 1:25 AM, Kevin Menard wrote:
>> So, if there is one thing that drives me nuts about Cayenne, it's
>> managing
>> ObjectContexts. In particular, you cannot relate two Persistent
>> objects
>> without first putting them into an ObjectContext. If one is
>> committed and
>> the other is not, you can have them in different contexts, but for
>> newly
>> created objects, this is a major pain in the neck.
>>
>> Since I've been complaining about it for probably close to three
>> years now,
>> I'd like to finally do something about it.
>>
>> Here are my rough notes from the airport:
>>
>> OK Cases:
>>
>> - Objects in same context
>> - Objects in different contexts, but objects are committed already
>>
>> Don't work, but should:
>>
>> - Objects in null contexts
>> - Objects in different contexts, but same data maps and domains
>>
>> Very hard to say, probably okay if don't work:
>>
>> - Objects in different contexts, contexts have different data maps
>> - Objects in different contexts, contexts have different data domains
>>
>>
>> As I started actually digging into the code, I ran into a lot of
>> NPE issues
>> trying to associate two Persistent objects with no context with
>> one another.
>> In an attempt to prevent adding special null-logic handling, I
>> thought about
>> applying the Null Object pattern to the problem. The basic idea
>> is rather
>> than use null as the default objectContext for CDO, use an
>> instance of
>> DelayedContext. The problem here is having objects in different
>> contexts.
>> So, it appears by fixing one, you can essentially fix the other.
>>
>> To address the latter, I was looking to have a Set of
>> ObjectContexts stored
>> in either BaseContext or DataContext. When willConnect() is
>> called, you'd
>> have something like the following:
>>
>> else if (this.getObjectContext().getEntityResolver() ==
>> object.getObjectContext().getEntityResolver()) {
>> ((DataContext)
>> this.getObjectContext()).addContextToMergeWith
>> (object.getObjectContext());
>> ((DataContext)
>> object.getObjectContext()).addContextToMergeWith
>> (this.getObjectContext());
>> }
>> else {
>> throw new CayenneRuntimeException(
>> "Cannot set object as destination of
>> relationship "
>> + relationshipName
>> + " because it is in a different
>> DataMap or
>> DataDomain.");
>> }
>>
>> (Casts are just an artifact of me screwing around).
>>
>> What I'm thinking would happen is that when commitChanges() is
>> called, the
>> set of contexts to be merged with will be iterated and any changes
>> applied
>> to the current object store / object store graph diff. The
>> relationship is
>> bidirectional so that the user can initiate the commit from any
>> object
>> registered with any context.
>>
>> Here is about where I lose it. I'm not as well-versed in the
>> internal
>> going-ons of Cayenne as I would like to be. It appears Cayenne
>> goes to
>> great efforts to essentially cache the graph manipulations so as
>> to avoid a
>> full traversal. I really don't know, though.
>>
>> Caching ordering of operations could make this tricky, but in
>> principal
>> should be wholly doable.
>>
>> If anyone has any thoughts on this or can fill in any missing
>> pieces, I'd
>> appreciate it. This is really something I'd like to see fixed
>> sooner rather
>> than later. I think it may be a requisite for JPA compliance as
>> well.
>>
>> --
>> Kevin
>>
>>
>
>
Re: Reconciling DataContexts
Posted by Kevin Menard <km...@servprise.com>.
On 10/21/07 4:20 PM, "Lachlan Deck" <la...@gmail.com> wrote:
>
> If this is something you'd prefer to do, Kevin, then I'd suggest
> adjusting your velocity templates to achieve what you'd like for the
> varying setter methods and so on. This is the usual place to do any
> such abstractions.
Certainly, that's doable. As is what Ari had suggested. The question is
should there be a better "default". If it is common practice for end users
to roll similar abstractions, that seems to be indicative of there needing
to be a better default. Unfortunately, that's a difficult thing to
ascertain.
--
Kevin
Re: Reconciling DataContexts
Posted by Lachlan Deck <la...@gmail.com>.
On 20/10/2007, at 11:40 PM, Kevin Menard wrote:
> On 10/20/07 4:36 AM, "Andrus Adamchik" <an...@objectstyle.org> wrote:
>
>> On Oct 19, 2007, at 10:58 PM, Kevin Menard wrote:
>>
>>> By hiding this in a setter, it may be possible to unregister
>>> the old
>>> object first, thereby limiting the growth.
>>
>> Not a good idea. Consider this:
>>
>> 1: BType blocal = (BType) a.getObjectContext().localObject
>> (b.getObjectId(), b);
>> 2: a.setSomething(blocal );
>> 3: a.setSomething( (BType) a.getObjectContext().localObject
>> (b1.getObjectId(), b1));
>> 4. blocal.doSomething();
>
> I guess I don't see that being distinctly different from the following
> non-Cayennish code:
>
> 1) BType b = a.getSomeB();
> 2) a.setSomeB(b1);
> 3) BType b1 = a.getSomeB();
> 4) b.doSomething();
If this is something you'd prefer to do, Kevin, then I'd suggest
adjusting your velocity templates to achieve what you'd like for the
varying setter methods and so on. This is the usual place to do any
such abstractions.
with regards,
--
Lachlan Deck
Re: Reconciling DataContexts
Posted by Andrus Adamchik <an...@objectstyle.org>.
On Oct 20, 2007, at 4:40 PM, Kevin Menard wrote:
> We've gone back and forth on this a few times now. As long as the
> conversation is still productive, I don't mind following it.
> Unfortunately,
> I don't have as great an understanding of the internals as you do,
> so if
> we're just rehashing old ground, we can agree to disagree. What
> I'd just
> hate to see is if we continue to do things the way we have been out
> of force
> of habit, rather than because it's the best way to do it. I see a
> lot of
> room for improvement and based on graph theory, can see how it
> could be
> solved. I haven't ascertained from the Cayenne code how disruptive
> a change
> it would be, however.
Yes, there seems to be a difference in approach to managing object
graphs... I feel like I do not understand many of your examples
simply cause I've been doing things differently, so I never thought
that certain things are actually problematic.
So I suspect that there is still misunderstanding on my end as to why
you are suggesting some of these changes. Can somebody else following
this discussion maybe point out what I am missing when defending the
existing design?
Andrus
Re: Reconciling DataContexts
Posted by Kevin Menard <km...@servprise.com>.
On 10/20/07 4:36 AM, "Andrus Adamchik" <an...@objectstyle.org> wrote:
>
> On Oct 19, 2007, at 10:58 PM, Kevin Menard wrote:
>
> I think actually the suggested terse syntax is more confusing on a
> number of levels:
>
> * normally a user would assume that after "a.setSomething(b)",
> "b.getSomethingElse() == a" (i.e. the reverse relationship is set),
> which would not be the case as reverse is set for the clone of "b".
> * and on a higher level it would obscure the fact that cross-context
> relationships are not possible by design.
Fair enough, but I've seen the problem play out in reverse as well. People
keep working with the original object, not the cloned. This gets strange as
that code starts getting distributed about. E.g., you have one method to
modify some type A and get it into a committed state. Another method where
you associate that A with B and maybe make some changes to it, but need not
commit them quite yet. Now you need to send that A out to every place that
you accessed the original A from.
Apologies if that seemed confusing. I can try to use specific cases if need
be. Suffice it to say, this is something that junior developers here have
done several times and it's not entirely fun to work out. Logically,
everything works the way it should. It's not until you consider how Cayenne
works that the problem becomes clear.
> I.e. I think explicit is good in this case. Also with JDK 1.5 switch
> we can remove the need for casts in many situations, making current
> approach less verbose.
Agreed. Although, I think having to grab the context and then calling
localObject() on that is still long-winded. If that be the case, I'd rather
see a new form of localObject() that takes a single Persistent and extracts
the object ID accordingly. I don't think I've ever seen the case where the
two parameters did not match in some way (save for using null as the second
one). It could be less verbose to move localObject() up to a utility method
of CDO itself as well.
>
>> 2) Loss of transient values.
>>
>> This could probably be addressed reflectively.
>
> True. Also JPA introduces explicit "transient" attributes - something
> we can use to address this issue.
I'll have to read about that. Is this something better addressed for JPA
first or for classic Cayenne first?
>
>> 3) Growth of object store.
>>
>> This is trickier. Ideally, if I call setSomething(a) with 50
>> different
>> instances of a, the object store would only have the latest a,
>> since that's
>> the only one that's going to be committed. What you have instead
>> is 50
>> different instances. With caching, I don't think DB access is the
>> issue,
>> but you will have an unbounded memory issue.
>
> I wouldn't worry about that in 3.0 - ObjectStore is using weak
> references now, so if you don't have a hard reference in the user
> code, the object will be gc'd as needed.
Okay. That's good to know. There is still a bit of a problem though,
because any changes made to any of the other 49 replaced objects objects
will still be committed and still potentially cause validation failures.
Maybe this should be expected, but what you have is an object that really
isn't attached to the graph at all any longer that will be committed. For
me, that creates for some confusing behavior. Debugging any validation
failures at that point is quite difficult, because your object graph could
validate fine but some cruft left in the object store is causing the
problem. If no one else views that as problematic though, I'll humbly
resign my stance on the matter.
>
>> By hiding this in a setter, it may be possible to unregister
>> the old
>> object first, thereby limiting the growth.
>
> Not a good idea. Consider this:
>
> 1: BType blocal = (BType) a.getObjectContext().localObject
> (b.getObjectId(), b);
> 2: a.setSomething(blocal );
> 3: a.setSomething( (BType) a.getObjectContext().localObject
> (b1.getObjectId(), b1));
> 4. blocal.doSomething();
I guess I don't see that being distinctly different from the following
non-Cayennish code:
1) BType b = a.getSomeB();
2) a.setSomeB(b1);
3) BType b1 = a.getSomeB();
4) b.doSomething();
What I do view as being problematic is if line 2 did not exist and the "b"
and "b1" are not the same object at all. This is sort of the situation we
have today.
In either case here, the call in line 2 sets the expectation that the
reference in line 1 is no longer overly useful (I'm sure someone can think
of a case where it is, but when working with a's latest b, it is not). One
could even draw a nice state diagram showing how this is the case and this
is not particular to Cayenne.
> "blocal" will be kicked out of the a's context in line 3, but the
> user already referenced this object in line 1, so when he later
> accesses it on line 4, it is no longer persistent. As I mentioned
> above there's no problem anymore with unused object accumulation, and
> generally trying to second-guess Cayenne on object graph management
> may cause undesired side effects.
Unfortunately, this doesn't appear to be the case. blocal will commit just
fine, which creates more problems to me than if the commit would fail
altogether.
We've gone back and forth on this a few times now. As long as the
conversation is still productive, I don't mind following it. Unfortunately,
I don't have as great an understanding of the internals as you do, so if
we're just rehashing old ground, we can agree to disagree. What I'd just
hate to see is if we continue to do things the way we have been out of force
of habit, rather than because it's the best way to do it. I see a lot of
room for improvement and based on graph theory, can see how it could be
solved. I haven't ascertained from the Cayenne code how disruptive a change
it would be, however.
--
Kevin
Re: Reconciling DataContexts
Posted by Andrus Adamchik <an...@objectstyle.org>.
On Oct 19, 2007, at 10:58 PM, Kevin Menard wrote:
> So, looking at the problem again, I see a few issues with
> localObject():
>
> 1) Overly verbose syntax. Consider:
>
> a.setSomething(b);
>
> versus
>
> a.setSomething((BType) a.getObjectContext().localObject
> (b.getObjectId(),
> b));
>
> I think this can largely be addressed by adding a new case to
> willConnect(), though. If the contexts are different, look at the
> persistence state and call localObject() automatically if it makes
> sense to.
> This helps ease up on the transparency issues as well. Rather than
> using
> one method for new, unregistered objects (setXXX()) and another for
> committed, registered objects (localObject()), you can consolidate
> to just
> the setter.
I think actually the suggested terse syntax is more confusing on a
number of levels:
* normally a user would assume that after "a.setSomething(b)",
"b.getSomethingElse() == a" (i.e. the reverse relationship is set),
which would not be the case as reverse is set for the clone of "b".
* and on a higher level it would obscure the fact that cross-context
relationships are not possible by design.
I.e. I think explicit is good in this case. Also with JDK 1.5 switch
we can remove the need for casts in many situations, making current
approach less verbose.
> 2) Loss of transient values.
>
> This could probably be addressed reflectively.
True. Also JPA introduces explicit "transient" attributes - something
we can use to address this issue.
> 3) Growth of object store.
>
> This is trickier. Ideally, if I call setSomething(a) with 50
> different
> instances of a, the object store would only have the latest a,
> since that's
> the only one that's going to be committed. What you have instead
> is 50
> different instances. With caching, I don't think DB access is the
> issue,
> but you will have an unbounded memory issue.
I wouldn't worry about that in 3.0 - ObjectStore is using weak
references now, so if you don't have a hard reference in the user
code, the object will be gc'd as needed.
> By hiding this in a setter, it may be possible to unregister
> the old
> object first, thereby limiting the growth.
Not a good idea. Consider this:
1: BType blocal = (BType) a.getObjectContext().localObject
(b.getObjectId(), b);
2: a.setSomething(blocal );
3: a.setSomething( (BType) a.getObjectContext().localObject
(b1.getObjectId(), b1));
4. blocal.doSomething();
"blocal" will be kicked out of the a's context in line 3, but the
user already referenced this object in line 1, so when he later
accesses it on line 4, it is no longer persistent. As I mentioned
above there's no problem anymore with unused object accumulation, and
generally trying to second-guess Cayenne on object graph management
may cause undesired side effects.
Andrus
Re: Reconciling DataContexts
Posted by Aristedes Maniatis <ar...@maniatis.org>.
On 20/10/2007, at 5:58 AM, Kevin Menard wrote:
> 1) Overly verbose syntax. Consider:
>
> a.setSomething(b);
>
> versus
>
> a.setSomething((BType) a.getObjectContext().localObject
> (b.getObjectId(),
> b));
I have a subclass of CayenneContext in our code with this helper method:
public Persistent localObject(Persistent po) {
if (po != null) {
if (po.getObjectContext() == this) {
return po;
}
return localObject(po.getObjectId(), null);
}
return null;
}
It doesn't completely solve your problem, but it does make the syntax
slightly shorter.
Perhaps another idea with an even simpler syntax for you is a pair of
functions in PersistentObject:
a.copyToContext(newContext)
a.copyToContext(b) // copies to the context of b
Personally I like the way contexts are managed in Cayenne. It is
always clear what context an object is in and there are no hidden
surprises. I can see what Kevin is getting at, but I think to obscure
the copying of objects between contexts will be less clear, not more.
Thinking about what context each object is in is part of working with
an ORM.
It would be great to have the context equivalent to generics: some
indicator which tells the programmer what context an object belongs
to and which doesn't let you (at compile time) relate objects from
different contexts. But I can't see how such a thing is possible.
Ari Maniatis
-------------------------->
Aristedes Maniatis
phone +61 2 9660 9700
PGP fingerprint 08 57 20 4B 80 69 59 E2 A9 BF 2D 48 C2 20 0C C8
Re: Reconciling DataContexts
Posted by Kevin Menard <km...@servprise.com>.
On 10/18/07 5:53 PM, "Kevin Menard" <km...@servprise.com> wrote:
>> That fact is *almost* transparent as relationships of the local
>> object are expanded as needed. What it doesn't cover is modified
>> objects in the graph. This is where we could use a method like JPA
>> 'merge' that would traverse already inflated graph and clone all
>> local changes for objects attached to a given object into the target
>> context.
>
> I'll have to double check this. I recall that not being the case, but it
> has been some time since I last tried.
Having looked at it again, I was mistaken. But, there is still a problem, I
just recalled it wrong. localObject() does not deal with transient
properties at all. So, anything you might put into your subclass basically
gets reset when you do a localObject() call. Also, the object store grows
with each localObject() call, rather than with the number of properties.
So, looking at the problem again, I see a few issues with localObject():
1) Overly verbose syntax. Consider:
a.setSomething(b);
versus
a.setSomething((BType) a.getObjectContext().localObject(b.getObjectId(),
b));
I think this can largely be addressed by adding a new case to
willConnect(), though. If the contexts are different, look at the
persistence state and call localObject() automatically if it makes sense to.
This helps ease up on the transparency issues as well. Rather than using
one method for new, unregistered objects (setXXX()) and another for
committed, registered objects (localObject()), you can consolidate to just
the setter.
2) Loss of transient values.
This could probably be addressed reflectively.
3) Growth of object store.
This is trickier. Ideally, if I call setSomething(a) with 50 different
instances of a, the object store would only have the latest a, since that's
the only one that's going to be committed. What you have instead is 50
different instances. With caching, I don't think DB access is the issue,
but you will have an unbounded memory issue.
By hiding this in a setter, it may be possible to unregister the old
object first, thereby limiting the growth.
I spent a decent amount of time looking into that, so I think the analysis
is fairly accurate. Feel free to correct if not though.
--
Kevin
Re: Reconciling DataContexts
Posted by Kevin Menard <km...@servprise.com>.
On 10/16/07 8:12 AM, "Andrus Adamchik" <an...@objectstyle.org> wrote:
>
> On Oct 16, 2007, at 2:57 PM, Kevin Menard wrote:
>> The problem with localObject is that it does not traverse the whole
>> graph,
>> making it practically useless in all but the most trivial situations.
>
> That fact is *almost* transparent as relationships of the local
> object are expanded as needed. What it doesn't cover is modified
> objects in the graph. This is where we could use a method like JPA
> 'merge' that would traverse already inflated graph and clone all
> local changes for objects attached to a given object into the target
> context.
I'll have to double check this. I recall that not being the case, but it
has been some time since I last tried.
>> What we've resorted to doing is committing the customer data, then
>> doing a
>> requery based on the ObjectID using the order context. This allows
>> us to
>> get a fully populated object. Of course, this means more trips to
>> the DB ...
>
> Not at all. All contexts work off of the same DataDomain cache, so
> lookup by ObjectId is fairly efficient. But it is not transparent, I
> agree.
This is probably on me then. I had to disable the cache because I was
experiencing corruption at one point. I also had another app that modified
the DB behind the scenes that was causing problems for the cache. With the
new cache implementations, I could probably deal with that better. I've
also begun moving that other app over to using ROP, which should improve the
shared cache situation.
So, in the case of not caching, it is a problem. But, you do have to
question why cache isn't being used in the first place.
>
>
>> I suppose the problem is exacerbated by the null context problem.
>> If we fix that, we'd likely be in much better shape.
>
> So it seems to me it comes down to implementing JPA-compatible
> "persist" and "merge" concepts on top of what we already have.
Could be. I think I only got through the first couple chapters of the JPA
spec. I'll have to print it out and give it a read.
--
Kevin
Re: Reconciling DataContexts
Posted by Andrus Adamchik <an...@objectstyle.org>.
On Oct 16, 2007, at 2:57 PM, Kevin Menard wrote:
> The problem with localObject is that it does not traverse the whole
> graph,
> making it practically useless in all but the most trivial situations.
That fact is *almost* transparent as relationships of the local
object are expanded as needed. What it doesn't cover is modified
objects in the graph. This is where we could use a method like JPA
'merge' that would traverse already inflated graph and clone all
local changes for objects attached to a given object into the target
context.
> Manually copying over the graph is doable, but a lot of work for
> something
> that really should be far easier.
>
> For example, we have customers and orders. All orders are
> associated with
> customers. We keep the customer info and order info in two different
> contexts for the most part, because we don't want to prematurely
> commit an
> order while a customer may feel it necessary to change his shipping
> address,
> for example.
>
> What we've resorted to doing is committing the customer data, then
> doing a
> requery based on the ObjectID using the order context. This allows
> us to
> get a fully populated object. Of course, this means more trips to
> the DB ...
Not at all. All contexts work off of the same DataDomain cache, so
lookup by ObjectId is fairly efficient. But it is not transparent, I
agree.
> I suppose the problem is exacerbated by the null context problem.
> If we fix that, we'd likely be in much better shape.
So it seems to me it comes down to implementing JPA-compatible
"persist" and "merge" concepts on top of what we already have.
Andrus
Re: Reconciling DataContexts
Posted by Kevin Menard <km...@servprise.com>.
The problem with localObject is that it does not traverse the whole graph,
making it practically useless in all but the most trivial situations.
Manually copying over the graph is doable, but a lot of work for something
that really should be far easier.
For example, we have customers and orders. All orders are associated with
customers. We keep the customer info and order info in two different
contexts for the most part, because we don't want to prematurely commit an
order while a customer may feel it necessary to change his shipping address,
for example.
What we've resorted to doing is committing the customer data, then doing a
requery based on the ObjectID using the order context. This allows us to
get a fully populated object. Of course, this means more trips to the DB
and a lot of committing/refetching code. It works, but it hardly makes
Cayenne transparent.
The whole thing becomes fragile, and there's a lot that Cayenne could do to
help out. I suppose the problem is exacerbated by the null context problem.
If we fix that, we'd likely be in much better shape. As it stands, with our
Tapestry app, we have to stage the objects at page render time to allow the
association of different objects. That creates for a lot of headaches as
well, and a lot of context management that really isn't necessary.
--
Kevin
On 10/16/07 9:02 AM, "Andrus Adamchik" <an...@objectstyle.org> wrote:
> Hi Kevin,
>
> Relating (as in "creating a direct reference in a Java sense") two
> objects belonging to two distinct contexts breaks fundamental Cayenne
> design, most notably assumptions about uniquing and transaction
> boundaries. I am -1 on that until somebody persuades me that this is
> a good idea and explains how to avoid messing up existing
> assumptions. (FWIW there is a workaround - referencing a peer of a
> given object in another context via 'localObject').
>
>
> On the other hand the inability to relate two TRANSIENT objects (i.e.
> objects without a context) is indeed a shortcuming. Here is one way
> how it might work (following the JPA patterns) :
>
> When such relationship is established, we would not attempt to create
> a reverse relationship (something that would require an
> EntityResolver to be present). We just create it one-way. So a user
> can build an object graph of an arbitrary size without a context and
> then at some point do one of the following with it:
>
> * "ObjectContext.registerNewObject(..)" (existing; equivalent to JPA
> "persist" method)
> * "ObjectContext.aNewMethod(..)" (does not exist yet; equivalent to
> JPA "merge" method and somewhat of an equivalent of a "localObject"
> method).
>
> Both would traverse a graph of transient objects (since they are not
> persistent yet, the graph is assumed to be finite and will not
> trigger sucking the entire DB in memory), attach them to the context
> and connect missing reverse relationships. The difference is that in
> the first case we'd assume that the objects are not yet persistent,
> while in a second case we'd attempt to match them against existing DB
> rows. (a typical use of a second case would be receiving XML-
> serialized stream of objects corresponding to the existing data).
>
>
> But you have something different in mind? Could you elaborate on the
> use cases?
>
> Thanks
> Andrus
>
>
> On Oct 16, 2007, at 1:25 AM, Kevin Menard wrote:
>> So, if there is one thing that drives me nuts about Cayenne, it's
>> managing
>> ObjectContexts. In particular, you cannot relate two Persistent
>> objects
>> without first putting them into an ObjectContext. If one is
>> committed and
>> the other is not, you can have them in different contexts, but for
>> newly
>> created objects, this is a major pain in the neck.
>>
>> Since I've been complaining about it for probably close to three
>> years now,
>> I'd like to finally do something about it.
>>
>> Here are my rough notes from the airport:
>>
>> OK Cases:
>>
>> - Objects in same context
>> - Objects in different contexts, but objects are committed already
>>
>> Don't work, but should:
>>
>> - Objects in null contexts
>> - Objects in different contexts, but same data maps and domains
>>
>> Very hard to say, probably okay if don't work:
>>
>> - Objects in different contexts, contexts have different data maps
>> - Objects in different contexts, contexts have different data domains
>>
>>
>> As I started actually digging into the code, I ran into a lot of
>> NPE issues
>> trying to associate two Persistent objects with no context with one
>> another.
>> In an attempt to prevent adding special null-logic handling, I
>> thought about
>> applying the Null Object pattern to the problem. The basic idea is
>> rather
>> than use null as the default objectContext for CDO, use an instance of
>> DelayedContext. The problem here is having objects in different
>> contexts.
>> So, it appears by fixing one, you can essentially fix the other.
>>
>> To address the latter, I was looking to have a Set of
>> ObjectContexts stored
>> in either BaseContext or DataContext. When willConnect() is
>> called, you'd
>> have something like the following:
>>
>> else if (this.getObjectContext().getEntityResolver() ==
>> object.getObjectContext().getEntityResolver()) {
>> ((DataContext)
>> this.getObjectContext()).addContextToMergeWith
>> (object.getObjectContext());
>> ((DataContext)
>> object.getObjectContext()).addContextToMergeWith
>> (this.getObjectContext());
>> }
>> else {
>> throw new CayenneRuntimeException(
>> "Cannot set object as destination of
>> relationship "
>> + relationshipName
>> + " because it is in a different
>> DataMap or
>> DataDomain.");
>> }
>>
>> (Casts are just an artifact of me screwing around).
>>
>> What I'm thinking would happen is that when commitChanges() is
>> called, the
>> set of contexts to be merged with will be iterated and any changes
>> applied
>> to the current object store / object store graph diff. The
>> relationship is
>> bidirectional so that the user can initiate the commit from any object
>> registered with any context.
>>
>> Here is about where I lose it. I'm not as well-versed in the internal
>> going-ons of Cayenne as I would like to be. It appears Cayenne
>> goes to
>> great efforts to essentially cache the graph manipulations so as to
>> avoid a
>> full traversal. I really don't know, though.
>>
>> Caching ordering of operations could make this tricky, but in
>> principal
>> should be wholly doable.
>>
>> If anyone has any thoughts on this or can fill in any missing
>> pieces, I'd
>> appreciate it. This is really something I'd like to see fixed
>> sooner rather
>> than later. I think it may be a requisite for JPA compliance as well.
>>
>> --
>> Kevin
>>
>>
>
--
Re: Reconciling DataContexts
Posted by Andrus Adamchik <an...@objectstyle.org>.
Hi Kevin,
Relating (as in "creating a direct reference in a Java sense") two
objects belonging to two distinct contexts breaks fundamental Cayenne
design, most notably assumptions about uniquing and transaction
boundaries. I am -1 on that until somebody persuades me that this is
a good idea and explains how to avoid messing up existing
assumptions. (FWIW there is a workaround - referencing a peer of a
given object in another context via 'localObject').
On the other hand the inability to relate two TRANSIENT objects (i.e.
objects without a context) is indeed a shortcuming. Here is one way
how it might work (following the JPA patterns) :
When such relationship is established, we would not attempt to create
a reverse relationship (something that would require an
EntityResolver to be present). We just create it one-way. So a user
can build an object graph of an arbitrary size without a context and
then at some point do one of the following with it:
* "ObjectContext.registerNewObject(..)" (existing; equivalent to JPA
"persist" method)
* "ObjectContext.aNewMethod(..)" (does not exist yet; equivalent to
JPA "merge" method and somewhat of an equivalent of a "localObject"
method).
Both would traverse a graph of transient objects (since they are not
persistent yet, the graph is assumed to be finite and will not
trigger sucking the entire DB in memory), attach them to the context
and connect missing reverse relationships. The difference is that in
the first case we'd assume that the objects are not yet persistent,
while in a second case we'd attempt to match them against existing DB
rows. (a typical use of a second case would be receiving XML-
serialized stream of objects corresponding to the existing data).
But you have something different in mind? Could you elaborate on the
use cases?
Thanks
Andrus
On Oct 16, 2007, at 1:25 AM, Kevin Menard wrote:
> So, if there is one thing that drives me nuts about Cayenne, it's
> managing
> ObjectContexts. In particular, you cannot relate two Persistent
> objects
> without first putting them into an ObjectContext. If one is
> committed and
> the other is not, you can have them in different contexts, but for
> newly
> created objects, this is a major pain in the neck.
>
> Since I've been complaining about it for probably close to three
> years now,
> I'd like to finally do something about it.
>
> Here are my rough notes from the airport:
>
> OK Cases:
>
> - Objects in same context
> - Objects in different contexts, but objects are committed already
>
> Don't work, but should:
>
> - Objects in null contexts
> - Objects in different contexts, but same data maps and domains
>
> Very hard to say, probably okay if don't work:
>
> - Objects in different contexts, contexts have different data maps
> - Objects in different contexts, contexts have different data domains
>
>
> As I started actually digging into the code, I ran into a lot of
> NPE issues
> trying to associate two Persistent objects with no context with one
> another.
> In an attempt to prevent adding special null-logic handling, I
> thought about
> applying the Null Object pattern to the problem. The basic idea is
> rather
> than use null as the default objectContext for CDO, use an instance of
> DelayedContext. The problem here is having objects in different
> contexts.
> So, it appears by fixing one, you can essentially fix the other.
>
> To address the latter, I was looking to have a Set of
> ObjectContexts stored
> in either BaseContext or DataContext. When willConnect() is
> called, you'd
> have something like the following:
>
> else if (this.getObjectContext().getEntityResolver() ==
> object.getObjectContext().getEntityResolver()) {
> ((DataContext)
> this.getObjectContext()).addContextToMergeWith
> (object.getObjectContext());
> ((DataContext)
> object.getObjectContext()).addContextToMergeWith
> (this.getObjectContext());
> }
> else {
> throw new CayenneRuntimeException(
> "Cannot set object as destination of
> relationship "
> + relationshipName
> + " because it is in a different
> DataMap or
> DataDomain.");
> }
>
> (Casts are just an artifact of me screwing around).
>
> What I'm thinking would happen is that when commitChanges() is
> called, the
> set of contexts to be merged with will be iterated and any changes
> applied
> to the current object store / object store graph diff. The
> relationship is
> bidirectional so that the user can initiate the commit from any object
> registered with any context.
>
> Here is about where I lose it. I'm not as well-versed in the internal
> going-ons of Cayenne as I would like to be. It appears Cayenne
> goes to
> great efforts to essentially cache the graph manipulations so as to
> avoid a
> full traversal. I really don't know, though.
>
> Caching ordering of operations could make this tricky, but in
> principal
> should be wholly doable.
>
> If anyone has any thoughts on this or can fill in any missing
> pieces, I'd
> appreciate it. This is really something I'd like to see fixed
> sooner rather
> than later. I think it may be a requisite for JPA compliance as well.
>
> --
> Kevin
>
>