You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cayenne.apache.org by "Musall, Maik" <ma...@selbstdenker.ag> on 2017/02/21 20:54:18 UTC
A way to refreshObject()
Hi all,
I have an application using a big shared snapshot cache. Objects freshly instantiated in an ObjectContext get their attributes populated based on the snapshot cache at the time, and keep them during the context's lifetime, which is also what I want. (Last week I found a page on cayenne.apache.org <http://cayenne.apache.org/> explaining this behaviour, but I can't find it again. Seems not to live within the linked Guide hierarchy.)
In some cases though, I have object instances in ObjectContext 1 which I know are getting modified in ObjectContext 2, and I want them to be updated after commit in context 1. In EOF, I could call oc1.refreshObject(obj), or even oc1.refreshAllObjects(), which will merge the new values in from the snapshot cache. Is there any way to do this in Cayenne, short of refetching which would cost a roundtrip to the database even though I know I already have the current values in the snapshot cache?
(I had a question posted recently about "creating a fault for an ObjectId", but this is different because I need the commit in context 2 to go through to the database.)
Maik
Re: A way to refreshObject()
Posted by Andrus Adamchik <an...@objectstyle.org>.
> On Feb 22, 2017, at 4:33 PM, Andrus Adamchik <an...@objectstyle.org> wrote:
>
>>>> oc1.refreshObject(obj), or even oc1.refreshAllObjects()
BTW, the following should probably work:
object.setPersistenceState(PersistenceState.HOLLOW);
As odd as it sounds, I've never tried doing it myself. But looking at the code, it should give the desired effect - getting data from shared cache on first property access, and bypassing the cached state at the ObjectContext level.
Andrus
Re: A way to refreshObject()
Posted by Andrus Adamchik <an...@objectstyle.org>.
> On Feb 22, 2017, at 10:42 PM, Musall, Maik <ma...@selbstdenker.ag> wrote:
>
>> IIRC this also whacks to-many relationships, and those require a query to get restored, as we can't guess relationship composition (another reason why I gave up on managing graph refreshing in memory).
>
> I did test adding a new object to a to-many relationship of an object, and that became visible after setting the root object HOLLOW, which is what I would expect.
Yes it will refresh, but will run a select query to do that.
Andrus
Re: A way to refreshObject()
Posted by "Musall, Maik" <ma...@selbstdenker.ag>.
> Am 22.02.2017 um 19:39 schrieb Andrus Adamchik <an...@objectstyle.org>:
>
>
>> On Feb 22, 2017, at 6:55 PM, Musall, Maik <ma...@selbstdenker.ag> wrote:
>>
>> Hi Andrus,
>>
>> ok, setting PersistenceState.HOLLOW works. So easy :-)
>
> Ah great. We all just learned a new Cayenne trick :)
>
> IIRC this also whacks to-many relationships, and those require a query to get restored, as we can't guess relationship composition (another reason why I gave up on managing graph refreshing in memory).
I did test adding a new object to a to-many relationship of an object, and that became visible after setting the root object HOLLOW, which is what I would expect. You think I need to do more testing on relationships regarding this?
>> However, I do have cases where I want to refresh the entire graph of objects in one ObjectContext, but I still want to make use of the snapshots to save the fetches if they tend to be expensive. This is how I'm doing it now, is this correct?
>>
>> for( Object obj : context.getGraphManager().registeredNodes() ) {
>> ((DataObject)obj).setPersistenceState( PersistenceState.HOLLOW );
>> }
>>
>> The assumption below all this of course is that I can only have one thread writing in one ObjectContext, although I can read on one ObjectContexts with several threads at once (like when using .parallelStream() on collections).
>
> This will work, but will not be atomic (your readers will be consuming data in the middle of invalidation ), and potentially will cycle through too many objects.
Well, in my case I can ensure that there is only one reader thread, which is waiting on the writing subthreads to complete execution, and which then will call the refreshAllObjects() itself. So, no problem there. (I did have to design the application so that only one thread would be operating on one editingContext in EOF at the time, even for read-only access.)
> Here is another idea. Assuming context #1 is where the changes happen, context #2 is where the reads occur. When you need a full refresh, create context #3, then transfer only the "entry point" objects from #2 via "localObject" and then switch the readers to #3 at some point and let the rest of the graph gradually inflate from cache. Again, no idea if that's workable in your setting. Logically readers should not care if they get a separate copy of the graph, as long as it is read-only. No?
Would work only if read access is really restricted to those entry point objects and anything traversed from there. It would be easy to introduce mistakes by holding on to references in #2 that didn't get the update. Also, activity in #2 would be limited to read-only operations even on objects that aren't touched by #1.
Maik
Re: A way to refreshObject()
Posted by Andrus Adamchik <an...@objectstyle.org>.
> On Feb 22, 2017, at 6:55 PM, Musall, Maik <ma...@selbstdenker.ag> wrote:
>
> Hi Andrus,
>
> ok, setting PersistenceState.HOLLOW works. So easy :-)
Ah great. We all just learned a new Cayenne trick :)
IIRC this also whacks to-many relationships, and those require a query to get restored, as we can't guess relationship composition (another reason why I gave up on managing graph refreshing in memory).
> I have cases where I have spent considerable resources to fetch large object graphs into memory, and then I'm sending off a few threads doing middle-duration asynchronous processing on a few objects, have them update an computed attribute and come back. When I'm receiving the message that object x is finished updating, it wouldn't make sense to refresh the entire object graph, let alone repeat any expensive db roundtrips.
Fair enough.
> However, I do have cases where I want to refresh the entire graph of objects in one ObjectContext, but I still want to make use of the snapshots to save the fetches if they tend to be expensive. This is how I'm doing it now, is this correct?
>
> for( Object obj : context.getGraphManager().registeredNodes() ) {
> ((DataObject)obj).setPersistenceState( PersistenceState.HOLLOW );
> }
>
> The assumption below all this of course is that I can only have one thread writing in one ObjectContext, although I can read on one ObjectContexts with several threads at once (like when using .parallelStream() on collections).
This will work, but will not be atomic (your readers will be consuming data in the middle of invalidation ), and potentially will cycle through too many objects.
Here is another idea. Assuming context #1 is where the changes happen, context #2 is where the reads occur. When you need a full refresh, create context #3, then transfer only the "entry point" objects from #2 via "localObject" and then switch the readers to #3 at some point and let the rest of the graph gradually inflate from cache. Again, no idea if that's workable in your setting. Logically readers should not care if they get a separate copy of the graph, as long as it is read-only. No?
Andrus
Re: A way to refreshObject()
Posted by "Musall, Maik" <ma...@selbstdenker.ag>.
Hi Andrus,
ok, setting PersistenceState.HOLLOW works. So easy :-)
I'm subscribing to your argument to generally refreshing entire collections for consistency. But this isn't about a simple CMS with a few blog posts. I have cases where I have spent considerable resources to fetch large object graphs into memory, and then I'm sending off a few threads doing middle-duration asynchronous processing on a few objects, have them update an computed attribute and come back. When I'm receiving the message that object x is finished updating, it wouldn't make sense to refresh the entire object graph, let alone repeat any expensive db roundtrips.
However, I do have cases where I want to refresh the entire graph of objects in one ObjectContext, but I still want to make use of the snapshots to save the fetches if they tend to be expensive. This is how I'm doing it now, is this correct?
for( Object obj : context.getGraphManager().registeredNodes() ) {
((DataObject)obj).setPersistenceState( PersistenceState.HOLLOW );
}
The assumption below all this of course is that I can only have one thread writing in one ObjectContext, although I can read on one ObjectContexts with several threads at once (like when using .parallelStream() on collections). Of course everything would be easier if the asynchronous threads could just all operate on the same ObjectContext and there would be no need to refresh anything in the first place.
And no, I don't want auto-synchronization. I do need the isolation of changes in the general case.
Maik
> Am 22.02.2017 um 14:33 schrieb Andrus Adamchik <an...@objectstyle.org>:
>
> Hi Maik,
>
> Like I said, I personally hate chasing individual object changes, and rather deal with refreshing entire collections. Refreshing individual objects quickly leads to cases that are nearly impossible to handle in the code, as modified objects may no longer match the criteria for being in a given collection.
>
> Say in a CMS you have a list of published articles. Another context changes published status of an article from true to false. You refreshed your object, but it is still in the published list. So now you need to filter that list in-memory. But the list still doesn't see any new articles that got published, so you need to track insert events, and so on.
>
> So instead of micro-optimizations with shared snapshot cache, you should use queries with cache groups and query cache. They get refetched when a cache group is flushed in response to an object commit. So technically you are not reusing the snapshot cache, but it really doesn't matter. The benefit you get in code simplicity, consistency, and often performance, always outweighs that. Besides this approach is cluster-friendly.
>
>> However, this page says "By default when a single ObjectContext commits its changes, all other contexts in the same runtime receive an event that contains all the committed changes", so perhaps there may be a way to send objectcontexts this event regardless?
>
> Not sure if you are convinced by the argument above :). But yeah, you can turn context syncing back on, and get auto-synchronization. So when you set Constants.SERVER_CONTEXTS_SYNC_PROPERTY to true (or rather don't set it to false), does it handle the refreshing the way that you want?
>
> Andrus
>
>
>> On Feb 22, 2017, at 4:02 PM, Musall, Maik <ma...@selbstdenker.ag> wrote:
>>
>> Hi Andrus,
>>
>> this works, but requires using the newly localObject()ed object instance to be used after refreshing. Any other references of that object in the original context will still hold the unchanged values.
>>
>> Btw, I found the page I mentioned: https://cayenne.apache.org/docs/4.0/cayenne-guide/performance-tuning.html#turning-off-synchronization-of-objectcontexts <https://cayenne.apache.org/docs/4.0/cayenne-guide/performance-tuning.html#turning-off-synchronization-of-objectcontexts>
>> I used the mentioned property setting in there to turn of synchronization of ObjectContexts, for the listed reasons. However, this page says "By default when a single ObjectContext commits its changes, all other contexts in the same runtime receive an event that contains all the committed changes", so perhaps there may be a way to send objectcontexts this event regardless?
>>
>> Maik
>>
>>
>>> Am 22.02.2017 um 07:37 schrieb Andrus Adamchik <an...@objectstyle.org>:
>>>
>>> Since we mostly focused on policy- and event-based *query* caches, the API for managing caching of individual objects is not as streamlined, but here it is FWIW:
>>>
>>> T myObject = ..;
>>> ObjectContext context = myObject.getObjectContext();
>>> ObjectId id = myObject.getObjectId();
>>>
>>> // kick it out, unset ObjectContext (side effect - sets ObjectId to null)
>>> context.getGraphManager().unregisterNode(id);
>>>
>>> // restore ObjectId.. it was set to null in the call above (which we probably should change)
>>> myObject.setObjectId(id);
>>>
>>> // get a fresh fault
>>> myObject = context.localObject(myObject);
>>>
>>> Andrus
>>>
>>>
>>>> On Feb 21, 2017, at 11:54 PM, Musall, Maik <ma...@selbstdenker.ag> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> I have an application using a big shared snapshot cache. Objects freshly instantiated in an ObjectContext get their attributes populated based on the snapshot cache at the time, and keep them during the context's lifetime, which is also what I want. (Last week I found a page on cayenne.apache.org <http://cayenne.apache.org/> explaining this behaviour, but I can't find it again. Seems not to live within the linked Guide hierarchy.)
>>>>
>>>> In some cases though, I have object instances in ObjectContext 1 which I know are getting modified in ObjectContext 2, and I want them to be updated after commit in context 1. In EOF, I could call oc1.refreshObject(obj), or even oc1.refreshAllObjects(), which will merge the new values in from the snapshot cache. Is there any way to do this in Cayenne, short of refetching which would cost a roundtrip to the database even though I know I already have the current values in the snapshot cache?
>>>>
>>>> (I had a question posted recently about "creating a fault for an ObjectId", but this is different because I need the commit in context 2 to go through to the database.)
>>>>
>>>> Maik
>
Re: A way to refreshObject()
Posted by Andrus Adamchik <an...@objectstyle.org>.
Hi Maik,
Like I said, I personally hate chasing individual object changes, and rather deal with refreshing entire collections. Refreshing individual objects quickly leads to cases that are nearly impossible to handle in the code, as modified objects may no longer match the criteria for being in a given collection.
Say in a CMS you have a list of published articles. Another context changes published status of an article from true to false. You refreshed your object, but it is still in the published list. So now you need to filter that list in-memory. But the list still doesn't see any new articles that got published, so you need to track insert events, and so on.
So instead of micro-optimizations with shared snapshot cache, you should use queries with cache groups and query cache. They get refetched when a cache group is flushed in response to an object commit. So technically you are not reusing the snapshot cache, but it really doesn't matter. The benefit you get in code simplicity, consistency, and often performance, always outweighs that. Besides this approach is cluster-friendly.
> However, this page says "By default when a single ObjectContext commits its changes, all other contexts in the same runtime receive an event that contains all the committed changes", so perhaps there may be a way to send objectcontexts this event regardless?
Not sure if you are convinced by the argument above :). But yeah, you can turn context syncing back on, and get auto-synchronization. So when you set Constants.SERVER_CONTEXTS_SYNC_PROPERTY to true (or rather don't set it to false), does it handle the refreshing the way that you want?
Andrus
> On Feb 22, 2017, at 4:02 PM, Musall, Maik <ma...@selbstdenker.ag> wrote:
>
> Hi Andrus,
>
> this works, but requires using the newly localObject()ed object instance to be used after refreshing. Any other references of that object in the original context will still hold the unchanged values.
>
> Btw, I found the page I mentioned: https://cayenne.apache.org/docs/4.0/cayenne-guide/performance-tuning.html#turning-off-synchronization-of-objectcontexts <https://cayenne.apache.org/docs/4.0/cayenne-guide/performance-tuning.html#turning-off-synchronization-of-objectcontexts>
> I used the mentioned property setting in there to turn of synchronization of ObjectContexts, for the listed reasons. However, this page says "By default when a single ObjectContext commits its changes, all other contexts in the same runtime receive an event that contains all the committed changes", so perhaps there may be a way to send objectcontexts this event regardless?
>
> Maik
>
>
>> Am 22.02.2017 um 07:37 schrieb Andrus Adamchik <an...@objectstyle.org>:
>>
>> Since we mostly focused on policy- and event-based *query* caches, the API for managing caching of individual objects is not as streamlined, but here it is FWIW:
>>
>> T myObject = ..;
>> ObjectContext context = myObject.getObjectContext();
>> ObjectId id = myObject.getObjectId();
>>
>> // kick it out, unset ObjectContext (side effect - sets ObjectId to null)
>> context.getGraphManager().unregisterNode(id);
>>
>> // restore ObjectId.. it was set to null in the call above (which we probably should change)
>> myObject.setObjectId(id);
>>
>> // get a fresh fault
>> myObject = context.localObject(myObject);
>>
>> Andrus
>>
>>
>>> On Feb 21, 2017, at 11:54 PM, Musall, Maik <ma...@selbstdenker.ag> wrote:
>>>
>>> Hi all,
>>>
>>> I have an application using a big shared snapshot cache. Objects freshly instantiated in an ObjectContext get their attributes populated based on the snapshot cache at the time, and keep them during the context's lifetime, which is also what I want. (Last week I found a page on cayenne.apache.org <http://cayenne.apache.org/> explaining this behaviour, but I can't find it again. Seems not to live within the linked Guide hierarchy.)
>>>
>>> In some cases though, I have object instances in ObjectContext 1 which I know are getting modified in ObjectContext 2, and I want them to be updated after commit in context 1. In EOF, I could call oc1.refreshObject(obj), or even oc1.refreshAllObjects(), which will merge the new values in from the snapshot cache. Is there any way to do this in Cayenne, short of refetching which would cost a roundtrip to the database even though I know I already have the current values in the snapshot cache?
>>>
>>> (I had a question posted recently about "creating a fault for an ObjectId", but this is different because I need the commit in context 2 to go through to the database.)
>>>
>>> Maik
Re: A way to refreshObject()
Posted by "Musall, Maik" <ma...@selbstdenker.ag>.
Hi Andrus,
this works, but requires using the newly localObject()ed object instance to be used after refreshing. Any other references of that object in the original context will still hold the unchanged values.
Btw, I found the page I mentioned: https://cayenne.apache.org/docs/4.0/cayenne-guide/performance-tuning.html#turning-off-synchronization-of-objectcontexts <https://cayenne.apache.org/docs/4.0/cayenne-guide/performance-tuning.html#turning-off-synchronization-of-objectcontexts>
I used the mentioned property setting in there to turn of synchronization of ObjectContexts, for the listed reasons. However, this page says "By default when a single ObjectContext commits its changes, all other contexts in the same runtime receive an event that contains all the committed changes", so perhaps there may be a way to send objectcontexts this event regardless?
Maik
> Am 22.02.2017 um 07:37 schrieb Andrus Adamchik <an...@objectstyle.org>:
>
> Since we mostly focused on policy- and event-based *query* caches, the API for managing caching of individual objects is not as streamlined, but here it is FWIW:
>
> T myObject = ..;
> ObjectContext context = myObject.getObjectContext();
> ObjectId id = myObject.getObjectId();
>
> // kick it out, unset ObjectContext (side effect - sets ObjectId to null)
> context.getGraphManager().unregisterNode(id);
>
> // restore ObjectId.. it was set to null in the call above (which we probably should change)
> myObject.setObjectId(id);
>
> // get a fresh fault
> myObject = context.localObject(myObject);
>
> Andrus
>
>
>> On Feb 21, 2017, at 11:54 PM, Musall, Maik <ma...@selbstdenker.ag> wrote:
>>
>> Hi all,
>>
>> I have an application using a big shared snapshot cache. Objects freshly instantiated in an ObjectContext get their attributes populated based on the snapshot cache at the time, and keep them during the context's lifetime, which is also what I want. (Last week I found a page on cayenne.apache.org <http://cayenne.apache.org/> explaining this behaviour, but I can't find it again. Seems not to live within the linked Guide hierarchy.)
>>
>> In some cases though, I have object instances in ObjectContext 1 which I know are getting modified in ObjectContext 2, and I want them to be updated after commit in context 1. In EOF, I could call oc1.refreshObject(obj), or even oc1.refreshAllObjects(), which will merge the new values in from the snapshot cache. Is there any way to do this in Cayenne, short of refetching which would cost a roundtrip to the database even though I know I already have the current values in the snapshot cache?
>>
>> (I had a question posted recently about "creating a fault for an ObjectId", but this is different because I need the commit in context 2 to go through to the database.)
>>
>> Maik
>>
>
Re: A way to refreshObject()
Posted by Andrus Adamchik <an...@objectstyle.org>.
Since we mostly focused on policy- and event-based *query* caches, the API for managing caching of individual objects is not as streamlined, but here it is FWIW:
T myObject = ..;
ObjectContext context = myObject.getObjectContext();
ObjectId id = myObject.getObjectId();
// kick it out, unset ObjectContext (side effect - sets ObjectId to null)
context.getGraphManager().unregisterNode(id);
// restore ObjectId.. it was set to null in the call above (which we probably should change)
myObject.setObjectId(id);
// get a fresh fault
myObject = context.localObject(myObject);
Andrus
> On Feb 21, 2017, at 11:54 PM, Musall, Maik <ma...@selbstdenker.ag> wrote:
>
> Hi all,
>
> I have an application using a big shared snapshot cache. Objects freshly instantiated in an ObjectContext get their attributes populated based on the snapshot cache at the time, and keep them during the context's lifetime, which is also what I want. (Last week I found a page on cayenne.apache.org <http://cayenne.apache.org/> explaining this behaviour, but I can't find it again. Seems not to live within the linked Guide hierarchy.)
>
> In some cases though, I have object instances in ObjectContext 1 which I know are getting modified in ObjectContext 2, and I want them to be updated after commit in context 1. In EOF, I could call oc1.refreshObject(obj), or even oc1.refreshAllObjects(), which will merge the new values in from the snapshot cache. Is there any way to do this in Cayenne, short of refetching which would cost a roundtrip to the database even though I know I already have the current values in the snapshot cache?
>
> (I had a question posted recently about "creating a fault for an ObjectId", but this is different because I need the commit in context 2 to go through to the database.)
>
> Maik
>