You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cayenne.apache.org by Andrus Adamchik <an...@objectstyle.org> on 2011/01/05 19:29:18 UTC
Re: transactions vs iterated query
Haven't thought about this scenario deeply... How about this:
In the simplest case you can keep your changes in the DataContext while iterating over the result (DataContext by itself is not permanently bound to a transaction). And then commit them after iteration is finished. This will work if only some objects have changes and you have enough memory to keep them in a DataContext.
Or if you need to flush data to DB in smaller chunks without committing a DB transaction, you can still do it by temporary re-bidning your own transaction to the current thread for the duration of commit, and then restoring it to the iterated query transaction when update is done.
Andrus
On Dec 29, 2010, at 3:25 PM, Hans Pikkemaat wrote:
> Hi,
>
> I'm using an iterated query to process a huge amount of data which cannot be loaded at once.
> This query creates its own transaction and binds it to the current thread.
>
> This means that when I process the data I received from the iterated query all queries use
> the transaction created by the iterated query.
>
> So any updates executed while the iterated query is running will be committed when the
> iterated query (actually the Result Iterator) is closed. So the iterator is committing the
> updates for me what I don't want.
>
> I know I can create my own transaction but this doesn't help me. I want to be able to
> create a transaction and within that transaction I want to run the iterated query and
> while in the query I want to do updates which are committed when I commit my own
> transaction.
>
> Question: Is there a way to prevent the iterated query to 'hyjack' my transaction?
>
> pseudo code
>
> create datacontext (my transaction)
> run iterated query (overrides my transaction)
> process data
> do some updates (will use the transaction of the iterator)
> close iterator (will commit my updates. I don't want this)
> commit my updates (does nothing as its already committed by the iterator)
>
> tx
>
> Hans
>
> PS: I'm using cayenne 2.0.4
>
Re: transactions vs iterated query
Posted by Hans Pikkemaat <h....@tsi-solutions.nl>.
Hi,
That sounds logical but the issue could be that the calling party is simply not aware that the
called library is using cayenne.
This would mean that an application which is calling some library always should clear
(and keep) the transaction. This is not very logical.
And what about the other example with the exception? Is there a way to call the close
method on the result iterator without exposing the ResultIterator to the calling party?
tx
Hans
On 1/6/11 10:08 AM, Andrus Adamchik wrote:
> On Jan 6, 2011, at 10:48 AM, Hans Pikkemaat wrote:
>
>> One library calls the other one.
>> The first one is using the iterated query to get some data. It will call the second
>> library to process the data.
> IMO this first library (iterator control code) should be the place that does transaction switching, transparently to the second library. E.g. in lib1:
>
> try {
> while(it.hasNext()) {
> Map nextRow = it.next();
>
> // save current tx in a local ivar
> try {
> // set current tx to NULL
> // call lib2
> }
> finally {
> // restore tx
> }
>
> }
> }
> finally {
> it.close();
> }
>
>> Now you could say that this is a design flaw.
> IMO it is.
>
> Andrus
Re: transactions vs iterated query
Posted by Andrus Adamchik <an...@objectstyle.org>.
On Jan 6, 2011, at 10:48 AM, Hans Pikkemaat wrote:
> One library calls the other one.
> The first one is using the iterated query to get some data. It will call the second
> library to process the data.
IMO this first library (iterator control code) should be the place that does transaction switching, transparently to the second library. E.g. in lib1:
try {
while(it.hasNext()) {
Map nextRow = it.next();
// save current tx in a local ivar
try {
// set current tx to NULL
// call lib2
}
finally {
// restore tx
}
}
}
finally {
it.close();
}
>
> Now you could say that this is a design flaw.
IMO it is.
Andrus
Re: transactions vs iterated query
Posted by Hans Pikkemaat <h....@tsi-solutions.nl>.
Hi,
After some investigation it comes down to two problems
1)
An example.
I have 2 libraries. One library calls the other one.
The first one is using the iterated query to get some data. It will call the second
library to process the data.
This second library however is not aware of a iterated query being used and it
simply creates a new DataContext and will commit it when it sees fit. But it doesn't
get a new transaction as there is already one bound to the thread.
This would effectively mean that every application that uses cayenne has to check
the bound transaction and if it exists create a new one and restore it later
(assuming he wants its own transaction). This is a weird situation.
2)
An example.
I have 2 libraries. One library calls the other one.
The first one is setting a task record to 'in progress'. Then it calls the second
library. The second one is using an iterated query but it is wrapped in an Iterator
such that the caller doesn't see the details. This however means that when an
exception occurs that the ResultIterator is not closed. This means that the transaction
is not unbound and as such never committed. The commit on the trasaction created
by the calling library then also doesn't work.
Now you could say that this is a design flaw. I partly agree. I should not wrap the
ResultIterator in an iterator as this prevents access to the close method.
So should I create a new iterator class with the close method and let this one
'bleed' through to the user? This also seems not a nice solution to me.
An thoughts?
tx
Hans
On 1/5/11 7:29 PM, Andrus Adamchik wrote:
> Haven't thought about this scenario deeply... How about this:
>
> In the simplest case you can keep your changes in the DataContext while iterating over the result (DataContext by itself is not permanently bound to a transaction). And then commit them after iteration is finished. This will work if only some objects have changes and you have enough memory to keep them in a DataContext.
>
> Or if you need to flush data to DB in smaller chunks without committing a DB transaction, you can still do it by temporary re-bidning your own transaction to the current thread for the duration of commit, and then restoring it to the iterated query transaction when update is done.
>
> Andrus
>
> On Dec 29, 2010, at 3:25 PM, Hans Pikkemaat wrote:
>> Hi,
>>
>> I'm using an iterated query to process a huge amount of data which cannot be loaded at once.
>> This query creates its own transaction and binds it to the current thread.
>>
>> This means that when I process the data I received from the iterated query all queries use
>> the transaction created by the iterated query.
>>
>> So any updates executed while the iterated query is running will be committed when the
>> iterated query (actually the Result Iterator) is closed. So the iterator is committing the
>> updates for me what I don't want.
>>
>> I know I can create my own transaction but this doesn't help me. I want to be able to
>> create a transaction and within that transaction I want to run the iterated query and
>> while in the query I want to do updates which are committed when I commit my own
>> transaction.
>>
>> Question: Is there a way to prevent the iterated query to 'hyjack' my transaction?
>>
>> pseudo code
>>
>> create datacontext (my transaction)
>> run iterated query (overrides my transaction)
>> process data
>> do some updates (will use the transaction of the iterator)
>> close iterator (will commit my updates. I don't want this)
>> commit my updates (does nothing as its already committed by the iterator)
>>
>> tx
>>
>> Hans
>>
>> PS: I'm using cayenne 2.0.4
>>