You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cayenne.apache.org by Bryan Lewis <br...@maine.rr.com> on 2010/07/20 15:05:30 UTC
timing issue?
We released a new internal web app at the start of the month and we've
noticed some troublesome behavior. After the insertion or deletion of an
object, a to-many relationship can occasionally get out of sync. For
example, we have a Company entity with a to-many relationship to Task. After
the user adds a task and returns to the task list page, the new object
doesn't appear. Sometimes. I've been working around it by adding explicit
refetches in the list pages.
I've tried to boil it down to a reproducible test case. The code below
simply adds and deletes a thousand objects and checks that the size of the
list is correct in a different DataContext. One of the assertions will fail
at some point in the test. I tried it with and without OSCache and the
error happens with both.
I wondered if it might be some kind of timing issue. If I added a
one-second pause whenever the lists disagreed, that was enough to get them
back in sync.
I'm not sure this completely explains the behavior we've been seeing. We
haven't been hitting the database that hard, with only about a half-dozen
users at any one time. Yesterday we had a case where one user's insertion
didn't appear in another user's list almost an hour later. It's a start for
discussion. It would be good news if I'm missing something.
private static final int cRuns = 1000;
private Employee user;
private RDTaskType taskType;
private org.apache.cayenne.access.DataContext createDataContext()
{
Configuration config = Configuration.getSharedConfiguration();
DataDomain domain = config.getDomain();
return domain.createDataContext();
}
void onActionFromTestCaching()
{
DataContext dc = createDataContext();
DataContext dc2 = createDataContext();
// Get a few objects needed to populate new tasks.
user = DataObjectUtils.objectForPK(dc, Employee.class, 312);
taskType = DataObjectUtils.objectForPK(dc, RDTaskType.class,
"CLIOR");
// A Company is the source of the to-many relationship. Get the
same
// one in two DataContexts.
Company company = DataObjectUtils.objectForPK(dc, Company.class,
5000);
Company company2 = (Company) dc2.localObject(company.getObjectId(),
null);
int nTasksStart = company.getTasks().size();
for (int i = 0; i < cRuns; i++) {
int nTasks = company.getTasks().size();
assert nTasks == nTasksStart + i : nTasks;
int nTasks2 = company2.getTasks().size();
if (nTasks2 != nTasks) {
debug("nTasks2 was " + nTasks2);
pause();
nTasks2 = company2.getTasks().size();
debug("nTasks2 now " + nTasks2);
}
assert nTasks2 == nTasks : "nTasks = " + nTasks + ", nTasks2 = "
+ nTasks2; // sometimes fails
insertTask(company);
}
List<Task> tasks = company.getTasks();
nTasksStart = tasks.size();
for (int i = 0; i < cRuns; i++) {
int nTasks = company.getTasks().size();
assert nTasks == nTasksStart - i : nTasks;
int nTasks2 = company2.getTasks().size();
if (nTasks2 != nTasks) {
debug("nTasks2 was " + nTasks2);
pause();
nTasks2 = company2.getTasks().size();
debug("nTasks2 now " + nTasks2);
}
assert nTasks2 == nTasks : "nTasks = " + nTasks + ", nTasks2 = "
+ nTasks2;
deleteTask(tasks.get(nTasks - 1));
}
}
private org.apache.cayenne.access.DataContext createDataContext()
{
Configuration config = Configuration.getSharedConfiguration();
DataDomain domain = config.getDomain();
return domain.createDataContext();
}
private void insertTask(Company company)
{
Date now = new java.util.Date();
DataContext dc = (DataContext) company.getObjectContext();
Task task = dc.newObject(Task.class);
task.setCreatedBy(user);
task.setLastModifiedBy(user);
task.setCreatedWhen(now);
task.setTargetDate(now);
task.setTimeWhen(now);
task.setCompany(company);
task.setAssignedTo(user);
user.addToTasks(task);
task.setType(taskType);
try {
dc.commitChanges();
}
catch (CayenneRuntimeException e) {
e.printStackTrace();
}
}
private void deleteTask(Task task)
{
DataContext dc = (DataContext) task.getObjectContext();
dc.deleteObject(task);
try {
dc.commitChanges();
}
catch (CayenneRuntimeException e) {
e.printStackTrace();
}
}
private long pauseMillis = 1000;
private void pause()
{
if (pauseMillis > 0) {
debug("pausing...");
try {
Thread.sleep(pauseMillis);
}
catch (InterruptedException ex) {
ex.printStackTrace();
}
}
}
Re: timing issue?
Posted by Andrus Adamchik <an...@objectstyle.org>.
Or you can use SelectQuery with prefetches for relationships and cache
groups plus entity listeners flushing appropriate cache groups on
commit. This requires more manual setup and planning upfront, but
better scales in terms of clustering, and gives better consistency
(not only individual objects are refreshed, but also all cached query
lists).
Andrus
On Jul 21, 2010, at 11:11 AM, Andrus Adamchik wrote:
> Hi Bryan,
>
> There can definitely be a small delay updating a peer context in the
> same JVM with the commit data. And this is probably what you are
> observing in the test (adding a delay gives all DataContexts a
> chance to digest the event). If the relationship hasn't been
> refreshed for an hour, this is caused by something else... My guess
> is that the Company object in a peer context had modifications of
> its own, so Cayenne bailed on merging the changes into it. There is
> a code in ObjectStore responsible for merging to-many:
>
> void processIndirectlyModifiedIDs(Collection indirectlyModifiedIDs) {
> Iterator indirectlyModifiedIt =
> indirectlyModifiedIDs.iterator();
> while (indirectlyModifiedIt.hasNext()) {
> ObjectId oid = (ObjectId) indirectlyModifiedIt.next();
>
> // access object map directly - the method should be
> called in a synchronized
> // context...
> final DataObject object = (DataObject) objectMap.get(oid);
>
> if (object == null
> || object.getPersistenceState() !=
> PersistenceState.COMMITTED) {
> continue;
> }
> ....
> }
>
> One possible way around it is to invalidate Company object after
> commit (then on next read it will refetch its relationship).
>
> Andrus
>
> On Jul 20, 2010, at 4:05 PM, Bryan Lewis wrote:
>
>> We released a new internal web app at the start of the month and
>> we've
>> noticed some troublesome behavior. After the insertion or deletion
>> of an
>> object, a to-many relationship can occasionally get out of sync. For
>> example, we have a Company entity with a to-many relationship to
>> Task. After
>> the user adds a task and returns to the task list page, the new
>> object
>> doesn't appear. Sometimes. I've been working around it by adding
>> explicit
>> refetches in the list pages.
>>
>> I've tried to boil it down to a reproducible test case. The code
>> below
>> simply adds and deletes a thousand objects and checks that the size
>> of the
>> list is correct in a different DataContext. One of the assertions
>> will fail
>> at some point in the test. I tried it with and without OSCache and
>> the
>> error happens with both.
>>
>> I wondered if it might be some kind of timing issue. If I added a
>> one-second pause whenever the lists disagreed, that was enough to
>> get them
>> back in sync.
>>
>> I'm not sure this completely explains the behavior we've been
>> seeing. We
>> haven't been hitting the database that hard, with only about a half-
>> dozen
>> users at any one time. Yesterday we had a case where one user's
>> insertion
>> didn't appear in another user's list almost an hour later. It's a
>> start for
>> discussion. It would be good news if I'm missing something.
>>
>>
>> private static final int cRuns = 1000;
>>
>> private Employee user;
>> private RDTaskType taskType;
>>
>> private org.apache.cayenne.access.DataContext createDataContext()
>> {
>> Configuration config = Configuration.getSharedConfiguration();
>> DataDomain domain = config.getDomain();
>> return domain.createDataContext();
>> }
>>
>> void onActionFromTestCaching()
>> {
>> DataContext dc = createDataContext();
>> DataContext dc2 = createDataContext();
>>
>> // Get a few objects needed to populate new tasks.
>> user = DataObjectUtils.objectForPK(dc, Employee.class, 312);
>> taskType = DataObjectUtils.objectForPK(dc, RDTaskType.class,
>> "CLIOR");
>>
>> // A Company is the source of the to-many relationship. Get the
>> same
>> // one in two DataContexts.
>> Company company = DataObjectUtils.objectForPK(dc, Company.class,
>> 5000);
>> Company company2 = (Company)
>> dc2.localObject(company.getObjectId(),
>> null);
>>
>> int nTasksStart = company.getTasks().size();
>>
>> for (int i = 0; i < cRuns; i++) {
>> int nTasks = company.getTasks().size();
>> assert nTasks == nTasksStart + i : nTasks;
>>
>> int nTasks2 = company2.getTasks().size();
>> if (nTasks2 != nTasks) {
>> debug("nTasks2 was " + nTasks2);
>> pause();
>> nTasks2 = company2.getTasks().size();
>> debug("nTasks2 now " + nTasks2);
>> }
>> assert nTasks2 == nTasks : "nTasks = " + nTasks + ",
>> nTasks2 = "
>> + nTasks2; // sometimes fails
>>
>> insertTask(company);
>> }
>>
>> List<Task> tasks = company.getTasks();
>> nTasksStart = tasks.size();
>>
>> for (int i = 0; i < cRuns; i++) {
>> int nTasks = company.getTasks().size();
>> assert nTasks == nTasksStart - i : nTasks;
>>
>> int nTasks2 = company2.getTasks().size();
>> if (nTasks2 != nTasks) {
>> debug("nTasks2 was " + nTasks2);
>> pause();
>> nTasks2 = company2.getTasks().size();
>> debug("nTasks2 now " + nTasks2);
>> }
>> assert nTasks2 == nTasks : "nTasks = " + nTasks + ",
>> nTasks2 = "
>> + nTasks2;
>>
>> deleteTask(tasks.get(nTasks - 1));
>> }
>> }
>>
>> private org.apache.cayenne.access.DataContext createDataContext()
>> {
>> Configuration config = Configuration.getSharedConfiguration();
>> DataDomain domain = config.getDomain();
>> return domain.createDataContext();
>> }
>>
>> private void insertTask(Company company)
>> {
>> Date now = new java.util.Date();
>> DataContext dc = (DataContext) company.getObjectContext();
>>
>> Task task = dc.newObject(Task.class);
>> task.setCreatedBy(user);
>> task.setLastModifiedBy(user);
>> task.setCreatedWhen(now);
>> task.setTargetDate(now);
>> task.setTimeWhen(now);
>> task.setCompany(company);
>> task.setAssignedTo(user);
>> user.addToTasks(task);
>> task.setType(taskType);
>>
>> try {
>> dc.commitChanges();
>> }
>> catch (CayenneRuntimeException e) {
>> e.printStackTrace();
>> }
>> }
>>
>> private void deleteTask(Task task)
>> {
>> DataContext dc = (DataContext) task.getObjectContext();
>> dc.deleteObject(task);
>>
>> try {
>> dc.commitChanges();
>> }
>> catch (CayenneRuntimeException e) {
>> e.printStackTrace();
>> }
>> }
>>
>> private long pauseMillis = 1000;
>>
>> private void pause()
>> {
>> if (pauseMillis > 0) {
>> debug("pausing...");
>> try {
>> Thread.sleep(pauseMillis);
>> }
>> catch (InterruptedException ex) {
>> ex.printStackTrace();
>> }
>> }
>> }
>
>
Re: timing issue?
Posted by Andrus Adamchik <an...@objectstyle.org>.
Hi Bryan,
There can definitely be a small delay updating a peer context in the
same JVM with the commit data. And this is probably what you are
observing in the test (adding a delay gives all DataContexts a chance
to digest the event). If the relationship hasn't been refreshed for an
hour, this is caused by something else... My guess is that the Company
object in a peer context had modifications of its own, so Cayenne
bailed on merging the changes into it. There is a code in ObjectStore
responsible for merging to-many:
void processIndirectlyModifiedIDs(Collection indirectlyModifiedIDs) {
Iterator indirectlyModifiedIt =
indirectlyModifiedIDs.iterator();
while (indirectlyModifiedIt.hasNext()) {
ObjectId oid = (ObjectId) indirectlyModifiedIt.next();
// access object map directly - the method should be
called in a synchronized
// context...
final DataObject object = (DataObject) objectMap.get(oid);
if (object == null
|| object.getPersistenceState() !=
PersistenceState.COMMITTED) {
continue;
}
....
}
One possible way around it is to invalidate Company object after
commit (then on next read it will refetch its relationship).
Andrus
On Jul 20, 2010, at 4:05 PM, Bryan Lewis wrote:
> We released a new internal web app at the start of the month and we've
> noticed some troublesome behavior. After the insertion or deletion
> of an
> object, a to-many relationship can occasionally get out of sync. For
> example, we have a Company entity with a to-many relationship to
> Task. After
> the user adds a task and returns to the task list page, the new object
> doesn't appear. Sometimes. I've been working around it by adding
> explicit
> refetches in the list pages.
>
> I've tried to boil it down to a reproducible test case. The code
> below
> simply adds and deletes a thousand objects and checks that the size
> of the
> list is correct in a different DataContext. One of the assertions
> will fail
> at some point in the test. I tried it with and without OSCache and
> the
> error happens with both.
>
> I wondered if it might be some kind of timing issue. If I added a
> one-second pause whenever the lists disagreed, that was enough to
> get them
> back in sync.
>
> I'm not sure this completely explains the behavior we've been
> seeing. We
> haven't been hitting the database that hard, with only about a half-
> dozen
> users at any one time. Yesterday we had a case where one user's
> insertion
> didn't appear in another user's list almost an hour later. It's a
> start for
> discussion. It would be good news if I'm missing something.
>
>
> private static final int cRuns = 1000;
>
> private Employee user;
> private RDTaskType taskType;
>
> private org.apache.cayenne.access.DataContext createDataContext()
> {
> Configuration config = Configuration.getSharedConfiguration();
> DataDomain domain = config.getDomain();
> return domain.createDataContext();
> }
>
> void onActionFromTestCaching()
> {
> DataContext dc = createDataContext();
> DataContext dc2 = createDataContext();
>
> // Get a few objects needed to populate new tasks.
> user = DataObjectUtils.objectForPK(dc, Employee.class, 312);
> taskType = DataObjectUtils.objectForPK(dc, RDTaskType.class,
> "CLIOR");
>
> // A Company is the source of the to-many relationship. Get the
> same
> // one in two DataContexts.
> Company company = DataObjectUtils.objectForPK(dc, Company.class,
> 5000);
> Company company2 = (Company)
> dc2.localObject(company.getObjectId(),
> null);
>
> int nTasksStart = company.getTasks().size();
>
> for (int i = 0; i < cRuns; i++) {
> int nTasks = company.getTasks().size();
> assert nTasks == nTasksStart + i : nTasks;
>
> int nTasks2 = company2.getTasks().size();
> if (nTasks2 != nTasks) {
> debug("nTasks2 was " + nTasks2);
> pause();
> nTasks2 = company2.getTasks().size();
> debug("nTasks2 now " + nTasks2);
> }
> assert nTasks2 == nTasks : "nTasks = " + nTasks + ",
> nTasks2 = "
> + nTasks2; // sometimes fails
>
> insertTask(company);
> }
>
> List<Task> tasks = company.getTasks();
> nTasksStart = tasks.size();
>
> for (int i = 0; i < cRuns; i++) {
> int nTasks = company.getTasks().size();
> assert nTasks == nTasksStart - i : nTasks;
>
> int nTasks2 = company2.getTasks().size();
> if (nTasks2 != nTasks) {
> debug("nTasks2 was " + nTasks2);
> pause();
> nTasks2 = company2.getTasks().size();
> debug("nTasks2 now " + nTasks2);
> }
> assert nTasks2 == nTasks : "nTasks = " + nTasks + ",
> nTasks2 = "
> + nTasks2;
>
> deleteTask(tasks.get(nTasks - 1));
> }
> }
>
> private org.apache.cayenne.access.DataContext createDataContext()
> {
> Configuration config = Configuration.getSharedConfiguration();
> DataDomain domain = config.getDomain();
> return domain.createDataContext();
> }
>
> private void insertTask(Company company)
> {
> Date now = new java.util.Date();
> DataContext dc = (DataContext) company.getObjectContext();
>
> Task task = dc.newObject(Task.class);
> task.setCreatedBy(user);
> task.setLastModifiedBy(user);
> task.setCreatedWhen(now);
> task.setTargetDate(now);
> task.setTimeWhen(now);
> task.setCompany(company);
> task.setAssignedTo(user);
> user.addToTasks(task);
> task.setType(taskType);
>
> try {
> dc.commitChanges();
> }
> catch (CayenneRuntimeException e) {
> e.printStackTrace();
> }
> }
>
> private void deleteTask(Task task)
> {
> DataContext dc = (DataContext) task.getObjectContext();
> dc.deleteObject(task);
>
> try {
> dc.commitChanges();
> }
> catch (CayenneRuntimeException e) {
> e.printStackTrace();
> }
> }
>
> private long pauseMillis = 1000;
>
> private void pause()
> {
> if (pauseMillis > 0) {
> debug("pausing...");
> try {
> Thread.sleep(pauseMillis);
> }
> catch (InterruptedException ex) {
> ex.printStackTrace();
> }
> }
> }