org.apache.derby.impl.store.raw.xact.Xact Destroy BUG?

[Susana Guedes <sg...@lasige.di.fc.ul.pt>]

Hi everyone,

I am working as a researcher in a European Investigation Project named 
GORDA - http://gorda.di.uminho.pt/
The goal is to make a patch to Derby to provide in-core replication 
mechanisms.
Currently i am reflecting the transaction context indicating a 
transaction Begin / Commit / Abort / Close

I think i found a bug in the method destroy() of 
org.apache.derby.impl.store.raw.xact.Xact class.

In this method you do:
if(state!=CLOSED){abort();}
close();

I think that you should be doing
if(state!=IDLE){abort();}
close();

Is this really a bug?  In my case the replicator is receiving a lot of 
ABORT notifications when he should only be receivig the CLOSE notification.


Regards
Susana

Re: org.apache.derby.impl.store.raw.xact.Xact Destroy BUG?

[Susana Guedes <sg...@lasige.di.fc.ul.pt>]

The GORDA project designed a novel database API to support replication.
This interface would be exported by database vendors, allowing 
replication protocols to be logically
decoupled from the database kernel, regardless how they are bundled in 
the final
product. Still, despite this logic separation, the API intends to expose
transaction processing structures and events such as parse trees, write 
sets,
or transactions, providing to the replicator the performance benefits of 
“in-core” monolithic
approaches.

Notice that in this way, my work of implementing this interfaces in 
derby will only work for the version i am currently using (v10.1.2.1) 
because its only a case study. But if you find it interesting, after i 
finish my case studie and produce a derby patch for version we can talk 
so that you can give some continuity to my work in the forthcoming 
versions. We are also implementing these interfaces in colaboration with 
PostgreSQL and MySql.

My email came in the sequence of the Transaction context interfaces 
defined that should expose the running transactions and respective 
information: version / id / isolation / respective database, connection 
and monitor / and also the most important state change notifications - 
begining, active, update, preparing, prepared, commiting, aborting, 
aborted, idle, closed.

I looked at derby and realized that the Xact class would be the most 
appropriate to give me access to all this information, in particular, to 
the state changes. I started by doing some prints of the state changes 
and found it hard to understand exactly when sould i notify a 'begining' 
and decided to do it when you invoque setNewTransactionId() in this 3 
diferent places: the Xact constructor, in getActiveStateTxIdString, and 
in setActiveState.

Regarding the commit/abort, i was expecting tha each transaction would 
lead to either one synchronized commit or to one abort, and that it 
would never happend that a synchronized commit could be followed by an 
abort. But that hapened because when you make a synchronized commit the 
state goes back to idle. Then, when you make a destroy(), you call an 
abort() if the state is other then closed. I think the described 
behaviour would be avoided if the destroy method only calls abort if the 
state is other that closed AND IDLE.

The test bed i used is
1) start network server
2) start one client that is based on SimpleClientNetworkSample but will 
do the following:
2.1) Create connection
2.2) Turn of auto-commit
2.3) Create a new statement using it to create an unexisting table
2.4) Commit
2.5) Close statement and connection

I can also attach the received output so that you can see the state 
changes of derby transactions and understand better what i have explained.


Susana

Mike Matrigali wrote:

> Can you add anything about what your approach with Derby will be?

> I suggest you search this list as there have been recent discussions 
> about
> replication schemes appropriate to Derby.
>
> I continue to think that starting at the Raw store level is the wrong
> place as the system is not designed to give a user level look at data
> at that point. And there are no open interfaces supported.
>
> One suggestion was using an open source product which captured the info
> at the jdbc level. Also others have used triggers to provide the
> necessary information.
>
> Can you make it clearer when you expect to receive an "ABORT" - for
> instance maybe only when user application either issues an abort or
> recieves a specific SQLState error? Derby generates a lot of internal
> aborts which either may never be seen by the user, or may be translated
> to other errors like a duplicate key error, or syntax error.
>
> Susana Guedes wrote:
>
>> Hi everyone,
>>
>> I am working as a researcher in a European Investigation Project 
>> named GORDA - http://gorda.di.uminho.pt/
>> The goal is to make a patch to Derby to provide in-core replication 
>> mechanisms.
>> Currently i am reflecting the transaction context indicating a 
>> transaction Begin / Commit / Abort / Close
>>
>> I think i found a bug in the method destroy() of 
>> org.apache.derby.impl.store.raw.xact.Xact class.
>>
>> In this method you do:
>> if(state!=CLOSED){abort();}
>> close();
>>
>> I think that you should be doing
>> if(state!=IDLE){abort();}
>> close();
>>
>> Is this really a bug? In my case the replicator is receiving a lot of 
>> ABORT notifications when he should only be receivig the CLOSE 
>> notification.
>>
>>
>> Regards
>> Susana
>>
>>

Re: org.apache.derby.impl.store.raw.xact.Xact Destroy BUG?

[Oystein Grovlen - Sun Norway <Oy...@Sun.COM>]

Mike Matrigali wrote:

> One suggestion was using an open source product which captured the info
> at the jdbc level.  Also others have used triggers to provide the
> necessary information.

One problem with statement-based replication (e.g., at JDBC level) is 
that statements are not idempotent.  This makes it difficult to do 
recovery on a replicated instance and to catch up with the other 
database instances.  After a crash one will normally have to copy the 
entire database instead of just doing a replay of what has happened 
while the instance was down. (Ref. Sequoia).  This would be easier with 
a row-based replication scheme.

With respect to trigger-based implementations, the biggest problem seems 
to be performance.  I do not know whether this is inherent with using 
triggers, but it seems like the overhead is pretty big in many database 
systems.

-- 
Øystein

Re: org.apache.derby.impl.store.raw.xact.Xact Destroy BUG?

[Mike Matrigali <mi...@sbcglobal.net>]

Can you add anything about what your approach with Derby will be?  I
suggest you search this list as there have been recent discussions about
replication schemes appropriate to Derby.

I continue to think that starting at the Raw store level is the wrong
place as the system is not designed to give a user level look at data
at that point.  And there are no open interfaces supported.

One suggestion was using an open source product which captured the info
at the jdbc level.  Also others have used triggers to provide the
necessary information.

Can you make it clearer when you expect to receive an "ABORT" - for
instance maybe only when user application either issues an abort or
recieves a specific SQLState error?  Derby generates a lot of internal
aborts which either may never be seen by the user, or may be translated
to other errors like a duplicate key error, or syntax error.

Susana Guedes wrote:
> Hi everyone,
> 
> I am working as a researcher in a European Investigation Project named 
> GORDA - http://gorda.di.uminho.pt/
> The goal is to make a patch to Derby to provide in-core replication 
> mechanisms.
> Currently i am reflecting the transaction context indicating a 
> transaction Begin / Commit / Abort / Close
> 
> I think i found a bug in the method destroy() of 
> org.apache.derby.impl.store.raw.xact.Xact class.
> 
> In this method you do:
> if(state!=CLOSED){abort();}
> close();
> 
> I think that you should be doing
> if(state!=IDLE){abort();}
> close();
> 
> Is this really a bug?  In my case the replicator is receiving a lot of 
> ABORT notifications when he should only be receivig the CLOSE notification.
> 
> 
> Regards
> Susana
> 
>