You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Cservenak Tamas <cs...@is-micro.hu> on 2005/11/22 13:28:18 UTC
[transaction] Duplicated TxId generation under heavy load
Hi all!
The generatedUniqueTxId() method in FileResourceManager uses
System.currentTimeMillis() to generate txId's.
On my system it causes duplicate txId generation and FRM failure. I have
4 threads accessing one FRM instance.
This simple patch adds "salt" to it, with a little overhead to solve
this problem.
tx
~t~
Index:
/home/cstamas/worx/projects/ext/eclipse/commons-transaction/src/java/org/apache/commons/transaction/file/FileResourceManager.java
===================================================================
---
/home/cstamas/worx/projects/ext/eclipse/commons-transaction/src/java/org/apache/commons/transaction/file/FileResourceManager.java
(revision 348140)
+++
/home/cstamas/worx/projects/ext/eclipse/commons-transaction/src/java/org/apache/commons/transaction/file/FileResourceManager.java
(working copy)
@@ -144,6 +144,9 @@
protected static final String WORK_DELETE_DIR = "delete";
protected static final String CONTEXT_FILE = "transaction.log";
+
+ // XXX used in txId generation
+ protected static long salt = 1;
/*
* --- Static helper methods ---
@@ -884,7 +887,8 @@
String txId;
synchronized (globalTransactions) {
do {
- txId = Long.toHexString(System.currentTimeMillis());
+ // XXX to prevent same txId generation on heavy load
+ txId = Long.toHexString(System.currentTimeMillis() +
salt++);
// XXX busy loop
} while (getContext(txId) != null);
}
Re: [transaction] Duplicated TxId generation under heavy load
Posted by Oliver Zeigermann <ol...@gmail.com>.
Cool.
Oliver
2005/11/22, Cservenak Tamas <cs...@is-micro.hu>:
> Hi,
>
> thank you for response. Jorg already convinced me to use some external Id
> generator, and not the handy built in (which was a handy solution for fast
> jump-in).
>
> My "patch" for generatedTxId will be simply moved to my code and it will do
> the trick, as it solves my problem.
>
> Thank you for help,
> Tamas
>
>
> Oliver Zeigermann wrote:
> To me it seems generatedUniqueTxId does exactly as advertised in
> Javadocs. Don't you agree?
>
> You simply need something different as it seems. Unique Id generators
> - that's what you need - are easy to find, even in the Jakarta Commons
> Project :)
>
> Oliver
>
> 2005/11/22, Cservenak Tamas <cs...@is-micro.hu>:
>
>
> Yes, youre right.
>
> Anyway, in my case it would be more appropriate to generate Tx ID's by
> myself to avoid generatedUniqueTxId() problem. As first step i will probably
> copy this patched method into my code and use it instead of calling
> frm.generated....
>
> Thanks for help. And sorry for bothering, this should go to the users
> maillist, not dev, right?
>
> ~t~
>
>
>
> Joerg Heinicke wrote:
> Cservenak Tamas <cservenak <at> is-micro.hu> writes:
>
>
>
> The generatedUniqueTxId() method in FileResourceManager uses
> System.currentTimeMillis() to generate txId's.
>
> On my system it causes duplicate txId generation and FRM failure. I have
> 4 threads accessing one FRM instance.
>
> This simple patch adds "salt" to it, with a little overhead to solve
> this problem.
>
> Unfortunately this does not help much, it only solves your problem "under
> heavy
> load". But if there is already another txId equal to this one (e.g.
> generated
> externally) FRM will again fail. It can only work if inside the synchronized
> block the generated txId is "preserved", e.g. by putting a final static
> object
> PRESERVED into the map and testing for it in the startTransaction(Object)
> method.
>
> This will solve almost all problems except one:
> 1. Thread 1: generateUniqueTxId()
> 2. Thread 2: startTransaction(txId) with an externally generated txId -
> coincidentally equal to the above generated one
> 3. Thread 1: startTransaction(txId) with the generated txId
>
> But is again a magnitude more unlikely.
>
> Jörg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> commons-dev-help@jakarta.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> commons-dev-help@jakarta.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> commons-dev-help@jakarta.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> commons-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [transaction] Duplicated TxId generation under heavy load
Posted by Cservenak Tamas <cs...@is-micro.hu>.
Hi,
thank you for response. Jorg already convinced me to use some external
Id generator, and not the handy built in (which was a handy solution for
fast jump-in).
My "patch" for generatedTxId will be simply moved to my code and it will
do the trick, as it solves my problem.
Thank you for help,
Tamas
Oliver Zeigermann wrote:
> To me it seems generatedUniqueTxId does exactly as advertised in
> Javadocs. Don't you agree?
>
> You simply need something different as it seems. Unique Id generators
> - that's what you need - are easy to find, even in the Jakarta Commons
> Project :)
>
> Oliver
>
> 2005/11/22, Cservenak Tamas <cs...@is-micro.hu>:
>
>> Yes, youre right.
>>
>> Anyway, in my case it would be more appropriate to generate Tx ID's by
>> myself to avoid generatedUniqueTxId() problem. As first step i will probably
>> copy this patched method into my code and use it instead of calling
>> frm.generated....
>>
>> Thanks for help. And sorry for bothering, this should go to the users
>> maillist, not dev, right?
>>
>> ~t~
>>
>>
>>
>> Joerg Heinicke wrote:
>> Cservenak Tamas <cservenak <at> is-micro.hu> writes:
>>
>>
>>
>> The generatedUniqueTxId() method in FileResourceManager uses
>> System.currentTimeMillis() to generate txId's.
>>
>> On my system it causes duplicate txId generation and FRM failure. I have
>> 4 threads accessing one FRM instance.
>>
>> This simple patch adds "salt" to it, with a little overhead to solve
>> this problem.
>>
>> Unfortunately this does not help much, it only solves your problem "under
>> heavy
>> load". But if there is already another txId equal to this one (e.g.
>> generated
>> externally) FRM will again fail. It can only work if inside the synchronized
>> block the generated txId is "preserved", e.g. by putting a final static
>> object
>> PRESERVED into the map and testing for it in the startTransaction(Object)
>> method.
>>
>> This will solve almost all problems except one:
>> 1. Thread 1: generateUniqueTxId()
>> 2. Thread 2: startTransaction(txId) with an externally generated txId -
>> coincidentally equal to the above generated one
>> 3. Thread 1: startTransaction(txId) with the generated txId
>>
>> But is again a magnitude more unlikely.
>>
>> Jörg
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> commons-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail:
>> commons-dev-help@jakarta.apache.org
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> commons-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail:
>> commons-dev-help@jakarta.apache.org
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
Re: [transaction] Duplicated TxId generation under heavy load
Posted by Oliver Zeigermann <ol...@gmail.com>.
Right. Thanks again for reporting that!
Should be fixed now.
Oliver
2006/1/13, Joerg Heinicke <jo...@gmx.de>:
> Oliver Zeigermann <oliver.zeigermann <at> gmail.com> writes:
>
> > See my latest commit for an - hopefully - acceptable degree of
> > certainty that the id really is unique.
>
> Hello Oliver,
>
> I reviewed the changes in commons-transaction since 1.1. And I want to come back
> to this old thread [1]. Your commit is at [2]. I wonder if there isn't missing a
> ++ on idCnt. Otherwise I guess the current solution is equal to the former one.
>
> WDYT?
>
> Jörg
>
> [1] http://marc.theaimsgroup.com/?t=113266273700001&r=1&w=4
> [2] http://svn.apache.org/viewcvs.cgi/jakarta/commons/proper/transaction/trunk/
> src/java/org/apache/commons/transaction/file/FileResourceManager.java
> ?r1=332340&r2=348458&rev=348458&sortby=date&diff_format=h
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [transaction] Duplicated TxId generation under heavy load
Posted by Joerg Heinicke <jo...@gmx.de>.
Oliver Zeigermann <oliver.zeigermann <at> gmail.com> writes:
> See my latest commit for an - hopefully - acceptable degree of
> certainty that the id really is unique.
Hello Oliver,
I reviewed the changes in commons-transaction since 1.1. And I want to come back
to this old thread [1]. Your commit is at [2]. I wonder if there isn't missing a
++ on idCnt. Otherwise I guess the current solution is equal to the former one.
WDYT?
Jörg
[1] http://marc.theaimsgroup.com/?t=113266273700001&r=1&w=4
[2] http://svn.apache.org/viewcvs.cgi/jakarta/commons/proper/transaction/trunk/
src/java/org/apache/commons/transaction/file/FileResourceManager.java
?r1=332340&r2=348458&rev=348458&sortby=date&diff_format=h
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [transaction] Duplicated TxId generation under heavy load
Posted by Oliver Zeigermann <ol...@gmail.com>.
See my latest commit for an - hopefully - acceptable degree of
certainty that the id really is unique. In the - very unlikely - case
that it really is not unique you can still catch the exception when
starting the tx and retry with a new one.
Acceptable?
Oliver
2005/11/23, Oliver Zeigermann <ol...@gmail.com>:
> 2005/11/23, Joerg Heinicke <jo...@gmx.de>:
> > Oliver Zeigermann <oliver.zeigermann <at> gmail.com> writes:
> >
> > > To me it seems generatedUniqueTxId does exactly as advertised in
> > > Javadocs. Don't you agree?
> >
> > No. :) This dismisses my argument about externally generated ids, yes. But two
> > different threads calling generateUniqueTxId() at the same time still get the
> > same "unique" id as the first thread calling this method does not "preserve" the
> > id. So the current implementation does not fulfill the "contract" mentioned in
> > the Javadocs.
>
> Argh! That's right, I guess :( Any idea how to solve this?
>
> Oliver
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [transaction] Duplicated TxId generation under heavy load
Posted by Oliver Zeigermann <ol...@gmail.com>.
2005/11/23, Joerg Heinicke <jo...@gmx.de>:
> Oliver Zeigermann <oliver.zeigermann <at> gmail.com> writes:
>
> > To me it seems generatedUniqueTxId does exactly as advertised in
> > Javadocs. Don't you agree?
>
> No. :) This dismisses my argument about externally generated ids, yes. But two
> different threads calling generateUniqueTxId() at the same time still get the
> same "unique" id as the first thread calling this method does not "preserve" the
> id. So the current implementation does not fulfill the "contract" mentioned in
> the Javadocs.
Argh! That's right, I guess :( Any idea how to solve this?
Oliver
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [transaction] Duplicated TxId generation under heavy load
Posted by Joerg Heinicke <jo...@gmx.de>.
Oliver Zeigermann <oliver.zeigermann <at> gmail.com> writes:
> To me it seems generatedUniqueTxId does exactly as advertised in
> Javadocs. Don't you agree?
No. :) This dismisses my argument about externally generated ids, yes. But two
different threads calling generateUniqueTxId() at the same time still get the
same "unique" id as the first thread calling this method does not "preserve" the
id. So the current implementation does not fulfill the "contract" mentioned in
the Javadocs.
> You simply need something different as it seems. Unique Id generators
> - that's what you need - are easy to find, even in the Jakarta Commons
> Project :)
For the externally generated ids I agree.
Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [transaction] Duplicated TxId generation under heavy load
Posted by Oliver Zeigermann <ol...@gmail.com>.
To me it seems generatedUniqueTxId does exactly as advertised in
Javadocs. Don't you agree?
You simply need something different as it seems. Unique Id generators
- that's what you need - are easy to find, even in the Jakarta Commons
Project :)
Oliver
2005/11/22, Cservenak Tamas <cs...@is-micro.hu>:
> Yes, youre right.
>
> Anyway, in my case it would be more appropriate to generate Tx ID's by
> myself to avoid generatedUniqueTxId() problem. As first step i will probably
> copy this patched method into my code and use it instead of calling
> frm.generated....
>
> Thanks for help. And sorry for bothering, this should go to the users
> maillist, not dev, right?
>
> ~t~
>
>
>
> Joerg Heinicke wrote:
> Cservenak Tamas <cservenak <at> is-micro.hu> writes:
>
>
>
> The generatedUniqueTxId() method in FileResourceManager uses
> System.currentTimeMillis() to generate txId's.
>
> On my system it causes duplicate txId generation and FRM failure. I have
> 4 threads accessing one FRM instance.
>
> This simple patch adds "salt" to it, with a little overhead to solve
> this problem.
>
> Unfortunately this does not help much, it only solves your problem "under
> heavy
> load". But if there is already another txId equal to this one (e.g.
> generated
> externally) FRM will again fail. It can only work if inside the synchronized
> block the generated txId is "preserved", e.g. by putting a final static
> object
> PRESERVED into the map and testing for it in the startTransaction(Object)
> method.
>
> This will solve almost all problems except one:
> 1. Thread 1: generateUniqueTxId()
> 2. Thread 2: startTransaction(txId) with an externally generated txId -
> coincidentally equal to the above generated one
> 3. Thread 1: startTransaction(txId) with the generated txId
>
> But is again a magnitude more unlikely.
>
> Jörg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> commons-dev-help@jakarta.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> commons-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [transaction] Duplicated TxId generation under heavy load
Posted by Cservenak Tamas <cs...@is-micro.hu>.
Yes, youre right.
Anyway, in my case it would be more appropriate to generate Tx ID's by
myself to avoid generatedUniqueTxId() problem. As first step i will
probably copy this patched method into my code and use it instead of
calling frm.generated....
Thanks for help. And sorry for bothering, this should go to the users
maillist, not dev, right?
~t~
Joerg Heinicke wrote:
> Cservenak Tamas <cservenak <at> is-micro.hu> writes:
>
>
>> The generatedUniqueTxId() method in FileResourceManager uses
>> System.currentTimeMillis() to generate txId's.
>>
>> On my system it causes duplicate txId generation and FRM failure. I have
>> 4 threads accessing one FRM instance.
>>
>> This simple patch adds "salt" to it, with a little overhead to solve
>> this problem.
>>
>
> Unfortunately this does not help much, it only solves your problem "under heavy
> load". But if there is already another txId equal to this one (e.g. generated
> externally) FRM will again fail. It can only work if inside the synchronized
> block the generated txId is "preserved", e.g. by putting a final static object
> PRESERVED into the map and testing for it in the startTransaction(Object) method.
>
> This will solve almost all problems except one:
> 1. Thread 1: generateUniqueTxId()
> 2. Thread 2: startTransaction(txId) with an externally generated txId -
> coincidentally equal to the above generated one
> 3. Thread 1: startTransaction(txId) with the generated txId
>
> But is again a magnitude more unlikely.
>
> Jörg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
Re: [transaction] Duplicated TxId generation under heavy load
Posted by Joerg Heinicke <jo...@gmx.de>.
Cservenak Tamas <cservenak <at> is-micro.hu> writes:
> The generatedUniqueTxId() method in FileResourceManager uses
> System.currentTimeMillis() to generate txId's.
>
> On my system it causes duplicate txId generation and FRM failure. I have
> 4 threads accessing one FRM instance.
>
> This simple patch adds "salt" to it, with a little overhead to solve
> this problem.
Unfortunately this does not help much, it only solves your problem "under heavy
load". But if there is already another txId equal to this one (e.g. generated
externally) FRM will again fail. It can only work if inside the synchronized
block the generated txId is "preserved", e.g. by putting a final static object
PRESERVED into the map and testing for it in the startTransaction(Object) method.
This will solve almost all problems except one:
1. Thread 1: generateUniqueTxId()
2. Thread 2: startTransaction(txId) with an externally generated txId -
coincidentally equal to the above generated one
3. Thread 1: startTransaction(txId) with the generated txId
But is again a magnitude more unlikely.
Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org