You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Frederic Toublanc <fr...@telemis.com> on 2013/05/13 10:37:21 UTC
Question about transactions
Hello everyone,
I have a question concerning transactions.
I dont find any answer in the documentation so here is the questions.
I start a transaction like this :
ds.begin(ReadWrite.write);
ds.getDefaultModel();
try {
ds.commit;
} catch (Exception ex) {
ds.abort();
} finally {
ds.end();}
If in the transction everything crashes so the transction is still active.
When i relaunch the application what is the state of the transaction that
wasn't ended (because of the crash) ?
Will i be able to start a new one with ds.begin ?
Re: Question about transactions
Posted by Patrick Logan <pa...@gmail.com>.
Thanks Andy. In some cases the data to load could be large enough that
I'd want to avoid one fell swoop with the transactional data
structures. Instead I was thinking it may be better to load a new
graph in the background with multiple, smaller transactions. Then make
the switch to avoid intervening readers from seeing the new data until
all the transactions are complete. For this particular application the
clients are not directly accessing the endpoint, so I already have an
intermediary that determines which endpoint and graph to use for a
given client.
On Wed, Jul 3, 2013 at 6:52 AM, Andy Seaborne <an...@apache.org> wrote:
> On 02/07/13 21:01, Patrick Logan wrote:
>>
>> I was looking into what behavior to expect in a related situation and
>> Andy's email here I think provides what I am looking for. Here is my
>> situation and application of this information:
>>
>> I would like a particular graph in fuseki to serve multiple readers
>> for some period of time. At some point an external process will come
>> along to refresh the data. In this case it will load an all-new graph,
>> verify things look good, and then indicate this new graph should be
>> used for subsequent queries.
>>
>> When this switch is made, that process will have fuseki delete the old
>> graph.
>>
>> Based on my previous understanding and Andy's email here, I believe
>> all reads in progress against the old graph will complete and only
>> then will the graph disappear for good.
>>
>> New queries that come along will be provided the name of the new
>> graph. (And I have some concurrency control around making the switch
>> between old and new graph names.)
>>
>> Does this seem reasonable?
>
>
> Yes.
>
> You don't need the "indicate this new graph should be used for subsequent
> queries." It just happens.
>
> When a writer commit, any newly starting query will get the new current
> state. Existing readers continue to see the old state.
>
> There are some scalability limits here - some of the transactional data
> structures are (currently) in-memory (JVM heap) with no spill to disk.
>
> It may be helpful to run with a default union graph and put new data into a
> named graph so it can be easily be all deleted at once later.
>
> Andy
>
>
>>
>> Thanks
>>
>>
>>
>>
>> On Mon, May 13, 2013 at 1:30 PM, Andy Seaborne <an...@apache.org> wrote:
>>>
>>> On 13/05/13 15:38, David Jordan wrote:
>>>>
>>>>
>>>> My understanding is that this may occur later, but better to let the
>>>> experts say for certain.
>>>
>>>
>>>
>>> You are exactly right. When used directly, TDB may buffer 10 commits
>>> before
>>> flushign to disk but the app does not see this - the code manages to know
>>> whether to look in the main DB or the journal.
>>>
>>> In Fuseki, there is little benfit to this and it writes back fro the
>>> journal
>>> to teh DB when ver it can.
>>>
>>> However, it can only write back if there is no READ transaction in
>>> progress.
>>> Such a read-txn needs the old data to be around so the journal can't
>>> overwrite it.
>>>
>>> If there is a long lived transaction, the journal keeps growing.
>>>
>>> Andy
>>>
>>>
>>>>
>>>> -----Original Message-----
>>>> From: Frederic Toublanc [mailto:frederic.toublanc@telemis.com]
>>>> Sent: Monday, May 13, 2013 10:34 AM
>>>> To: users@jena.apache.org
>>>> Subject: Re: Question about transactions
>>>>
>>>> Ok i dont undestand.
>>>>
>>>> I though that ds.commit() really updated the database files...
>>>>
>>>>
>>>> 2013/5/13 David Jordan <Da...@sas.com>
>>>>
>>>>> I assume the journal file will shrink in size once the updates have
>>>>> actually been applied to the database files?
>>>>> If no transactions are in progress, is there a way to initiate this
>>>>> transfer of data from the journal to the database?
>>>>> It may also be of interest to the application to hold off this
>>>>> transfer from the journal to the database until a certain point in
>>>>> time.
>>>>>
>>>>> -----Original Message-----
>>>>> From: Frederic Toublanc [mailto:frederic.toublanc@telemis.com]
>>>>> Sent: Monday, May 13, 2013 10:08 AM
>>>>> To: users@jena.apache.org
>>>>> Subject: Re: Question about transactions
>>>>>
>>>>> Ok now i have an other problem.
>>>>> Each time i end i write transaction the size of the TDB keeps growing
>>>>> because of the file journal.jrnl.
>>>>>
>>>>>
>>>>> 2013/5/13 Frederic Toublanc <fr...@telemis.com>
>>>>>
>>>>>> Ok thx a lot :)
>>>>>>
>>>>>> Yes by "everythin" i meant that the JVM dies or a power failure.
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013/5/13 Andy Seaborne <an...@apache.org>
>>>>>>
>>>>>>> On 13/05/13 09:37, Frederic Toublanc wrote:
>>>>>>>
>>>>>>>> Hello everyone,
>>>>>>>>
>>>>>>>> I have a question concerning transactions.
>>>>>>>> I dont find any answer in the documentation so here is the
>>>>>>>> questions.
>>>>>>>>
>>>>>>>> I start a transaction like this :
>>>>>>>>
>>>>>>>> ds.begin(ReadWrite.write);
>>>>>>>>
>>>>>>>> ds.getDefaultModel();
>>>>>>>>
>>>>>>>> try {
>>>>>>>>
>>>>>>>> ds.commit;
>>>>>>>> } catch (Exception ex) {
>>>>>>>>
>>>>>>>
>>>>>>> catch (Throwable ex) is slightly better.
>>>>>>>
>>>>>>> ds.abort();
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> } finally {
>>>>>>>>
>>>>>>>>
>>>>>>>> ds.end();}
>>>>>>>>
>>>>>>>> If in the transction everything crashes so the transction is still
>>>>>>>> active.
>>>>>>>>
>>>>>>>
>>>>>>> By "everything" you mean the JVM dies suddenly and unexpectedly?
>>>>>>> Power failure, whale falling on data centre (yes - it has
>>>>>>> happened!)
>>>>>>>
>>>>>>> TDB does assume fail-stop of hardware and OS - no ongoing random
>>>>>>> partial failures or silently corrupting RAM although it does
>>>>>>> checksum the transaction log entries.
>>>>>>>
>>>>>>> On restart transaction is either commited or not.
>>>>>>>
>>>>>>> If the machine crashed before the commit was called, then the
>>>>>>> transaction is was aborted (by simply forgetting about it - TDB
>>>>>>> does not change the core databased as the application progress - it
>>>>>>> uses
>>>>>
>>>>>
>>>>> journalling.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> If the machine crashed after the commit() -- even if before end()
>>>>>>> -- the transaction committed and will be recovered on next startup.
>>>>>>>
>>>>>>> If the machine crashed during commit then the transaction either
>>>>>>> completely committed or not all (and will be aborted). There is no
>>>>>>> "half-committed". The true internal commit point is the sync of a
>>>>>>> single disk block.
>>>>>>>
>>>>>>> Your code may have to check whether the transaction happened or not
>>>>>>> - any transaction system has a point where a commit has been made
>>>>>>> but before the application is informed.
>>>>>>>
>>>>>>>
>>>>>>> When i relaunch the application what is the state of the
>>>>>>> transaction that
>>>>>>>>
>>>>>>>>
>>>>>>>> wasn't ended (because of the crash) ?
>>>>>>>> Will i be able to start a new one with ds.begin ?
>>>>>>>>
>>>>>>>
>>>>>>> Yes.
>>>>>>>
>>>>>>> Andy
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>
Re: Question about transactions
Posted by Andy Seaborne <an...@apache.org>.
On 02/07/13 21:01, Patrick Logan wrote:
> I was looking into what behavior to expect in a related situation and
> Andy's email here I think provides what I am looking for. Here is my
> situation and application of this information:
>
> I would like a particular graph in fuseki to serve multiple readers
> for some period of time. At some point an external process will come
> along to refresh the data. In this case it will load an all-new graph,
> verify things look good, and then indicate this new graph should be
> used for subsequent queries.
>
> When this switch is made, that process will have fuseki delete the old graph.
>
> Based on my previous understanding and Andy's email here, I believe
> all reads in progress against the old graph will complete and only
> then will the graph disappear for good.
>
> New queries that come along will be provided the name of the new
> graph. (And I have some concurrency control around making the switch
> between old and new graph names.)
>
> Does this seem reasonable?
Yes.
You don't need the "indicate this new graph should be used for
subsequent queries." It just happens.
When a writer commit, any newly starting query will get the new current
state. Existing readers continue to see the old state.
There are some scalability limits here - some of the transactional data
structures are (currently) in-memory (JVM heap) with no spill to disk.
It may be helpful to run with a default union graph and put new data
into a named graph so it can be easily be all deleted at once later.
Andy
>
> Thanks
>
>
>
>
> On Mon, May 13, 2013 at 1:30 PM, Andy Seaborne <an...@apache.org> wrote:
>> On 13/05/13 15:38, David Jordan wrote:
>>>
>>> My understanding is that this may occur later, but better to let the
>>> experts say for certain.
>>
>>
>> You are exactly right. When used directly, TDB may buffer 10 commits before
>> flushign to disk but the app does not see this - the code manages to know
>> whether to look in the main DB or the journal.
>>
>> In Fuseki, there is little benfit to this and it writes back fro the journal
>> to teh DB when ver it can.
>>
>> However, it can only write back if there is no READ transaction in progress.
>> Such a read-txn needs the old data to be around so the journal can't
>> overwrite it.
>>
>> If there is a long lived transaction, the journal keeps growing.
>>
>> Andy
>>
>>
>>>
>>> -----Original Message-----
>>> From: Frederic Toublanc [mailto:frederic.toublanc@telemis.com]
>>> Sent: Monday, May 13, 2013 10:34 AM
>>> To: users@jena.apache.org
>>> Subject: Re: Question about transactions
>>>
>>> Ok i dont undestand.
>>>
>>> I though that ds.commit() really updated the database files...
>>>
>>>
>>> 2013/5/13 David Jordan <Da...@sas.com>
>>>
>>>> I assume the journal file will shrink in size once the updates have
>>>> actually been applied to the database files?
>>>> If no transactions are in progress, is there a way to initiate this
>>>> transfer of data from the journal to the database?
>>>> It may also be of interest to the application to hold off this
>>>> transfer from the journal to the database until a certain point in time.
>>>>
>>>> -----Original Message-----
>>>> From: Frederic Toublanc [mailto:frederic.toublanc@telemis.com]
>>>> Sent: Monday, May 13, 2013 10:08 AM
>>>> To: users@jena.apache.org
>>>> Subject: Re: Question about transactions
>>>>
>>>> Ok now i have an other problem.
>>>> Each time i end i write transaction the size of the TDB keeps growing
>>>> because of the file journal.jrnl.
>>>>
>>>>
>>>> 2013/5/13 Frederic Toublanc <fr...@telemis.com>
>>>>
>>>>> Ok thx a lot :)
>>>>>
>>>>> Yes by "everythin" i meant that the JVM dies or a power failure.
>>>>>
>>>>>
>>>>>
>>>>> 2013/5/13 Andy Seaborne <an...@apache.org>
>>>>>
>>>>>> On 13/05/13 09:37, Frederic Toublanc wrote:
>>>>>>
>>>>>>> Hello everyone,
>>>>>>>
>>>>>>> I have a question concerning transactions.
>>>>>>> I dont find any answer in the documentation so here is the questions.
>>>>>>>
>>>>>>> I start a transaction like this :
>>>>>>>
>>>>>>> ds.begin(ReadWrite.write);
>>>>>>>
>>>>>>> ds.getDefaultModel();
>>>>>>>
>>>>>>> try {
>>>>>>>
>>>>>>> ds.commit;
>>>>>>> } catch (Exception ex) {
>>>>>>>
>>>>>>
>>>>>> catch (Throwable ex) is slightly better.
>>>>>>
>>>>>> ds.abort();
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> } finally {
>>>>>>>
>>>>>>> ds.end();}
>>>>>>>
>>>>>>> If in the transction everything crashes so the transction is still
>>>>>>> active.
>>>>>>>
>>>>>>
>>>>>> By "everything" you mean the JVM dies suddenly and unexpectedly?
>>>>>> Power failure, whale falling on data centre (yes - it has
>>>>>> happened!)
>>>>>>
>>>>>> TDB does assume fail-stop of hardware and OS - no ongoing random
>>>>>> partial failures or silently corrupting RAM although it does
>>>>>> checksum the transaction log entries.
>>>>>>
>>>>>> On restart transaction is either commited or not.
>>>>>>
>>>>>> If the machine crashed before the commit was called, then the
>>>>>> transaction is was aborted (by simply forgetting about it - TDB
>>>>>> does not change the core databased as the application progress - it
>>>>>> uses
>>>>
>>>> journalling.
>>>>>>
>>>>>>
>>>>>> If the machine crashed after the commit() -- even if before end()
>>>>>> -- the transaction committed and will be recovered on next startup.
>>>>>>
>>>>>> If the machine crashed during commit then the transaction either
>>>>>> completely committed or not all (and will be aborted). There is no
>>>>>> "half-committed". The true internal commit point is the sync of a
>>>>>> single disk block.
>>>>>>
>>>>>> Your code may have to check whether the transaction happened or not
>>>>>> - any transaction system has a point where a commit has been made
>>>>>> but before the application is informed.
>>>>>>
>>>>>>
>>>>>> When i relaunch the application what is the state of the
>>>>>> transaction that
>>>>>>>
>>>>>>> wasn't ended (because of the crash) ?
>>>>>>> Will i be able to start a new one with ds.begin ?
>>>>>>>
>>>>>>
>>>>>> Yes.
>>>>>>
>>>>>> Andy
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
Re: Question about transactions
Posted by Patrick Logan <pa...@gmail.com>.
I was looking into what behavior to expect in a related situation and
Andy's email here I think provides what I am looking for. Here is my
situation and application of this information:
I would like a particular graph in fuseki to serve multiple readers
for some period of time. At some point an external process will come
along to refresh the data. In this case it will load an all-new graph,
verify things look good, and then indicate this new graph should be
used for subsequent queries.
When this switch is made, that process will have fuseki delete the old graph.
Based on my previous understanding and Andy's email here, I believe
all reads in progress against the old graph will complete and only
then will the graph disappear for good.
New queries that come along will be provided the name of the new
graph. (And I have some concurrency control around making the switch
between old and new graph names.)
Does this seem reasonable?
Thanks
On Mon, May 13, 2013 at 1:30 PM, Andy Seaborne <an...@apache.org> wrote:
> On 13/05/13 15:38, David Jordan wrote:
>>
>> My understanding is that this may occur later, but better to let the
>> experts say for certain.
>
>
> You are exactly right. When used directly, TDB may buffer 10 commits before
> flushign to disk but the app does not see this - the code manages to know
> whether to look in the main DB or the journal.
>
> In Fuseki, there is little benfit to this and it writes back fro the journal
> to teh DB when ver it can.
>
> However, it can only write back if there is no READ transaction in progress.
> Such a read-txn needs the old data to be around so the journal can't
> overwrite it.
>
> If there is a long lived transaction, the journal keeps growing.
>
> Andy
>
>
>>
>> -----Original Message-----
>> From: Frederic Toublanc [mailto:frederic.toublanc@telemis.com]
>> Sent: Monday, May 13, 2013 10:34 AM
>> To: users@jena.apache.org
>> Subject: Re: Question about transactions
>>
>> Ok i dont undestand.
>>
>> I though that ds.commit() really updated the database files...
>>
>>
>> 2013/5/13 David Jordan <Da...@sas.com>
>>
>>> I assume the journal file will shrink in size once the updates have
>>> actually been applied to the database files?
>>> If no transactions are in progress, is there a way to initiate this
>>> transfer of data from the journal to the database?
>>> It may also be of interest to the application to hold off this
>>> transfer from the journal to the database until a certain point in time.
>>>
>>> -----Original Message-----
>>> From: Frederic Toublanc [mailto:frederic.toublanc@telemis.com]
>>> Sent: Monday, May 13, 2013 10:08 AM
>>> To: users@jena.apache.org
>>> Subject: Re: Question about transactions
>>>
>>> Ok now i have an other problem.
>>> Each time i end i write transaction the size of the TDB keeps growing
>>> because of the file journal.jrnl.
>>>
>>>
>>> 2013/5/13 Frederic Toublanc <fr...@telemis.com>
>>>
>>>> Ok thx a lot :)
>>>>
>>>> Yes by "everythin" i meant that the JVM dies or a power failure.
>>>>
>>>>
>>>>
>>>> 2013/5/13 Andy Seaborne <an...@apache.org>
>>>>
>>>>> On 13/05/13 09:37, Frederic Toublanc wrote:
>>>>>
>>>>>> Hello everyone,
>>>>>>
>>>>>> I have a question concerning transactions.
>>>>>> I dont find any answer in the documentation so here is the questions.
>>>>>>
>>>>>> I start a transaction like this :
>>>>>>
>>>>>> ds.begin(ReadWrite.write);
>>>>>>
>>>>>> ds.getDefaultModel();
>>>>>>
>>>>>> try {
>>>>>>
>>>>>> ds.commit;
>>>>>> } catch (Exception ex) {
>>>>>>
>>>>>
>>>>> catch (Throwable ex) is slightly better.
>>>>>
>>>>> ds.abort();
>>>>>>
>>>>>>
>>>>>
>>>>> } finally {
>>>>>>
>>>>>> ds.end();}
>>>>>>
>>>>>> If in the transction everything crashes so the transction is still
>>>>>> active.
>>>>>>
>>>>>
>>>>> By "everything" you mean the JVM dies suddenly and unexpectedly?
>>>>> Power failure, whale falling on data centre (yes - it has
>>>>> happened!)
>>>>>
>>>>> TDB does assume fail-stop of hardware and OS - no ongoing random
>>>>> partial failures or silently corrupting RAM although it does
>>>>> checksum the transaction log entries.
>>>>>
>>>>> On restart transaction is either commited or not.
>>>>>
>>>>> If the machine crashed before the commit was called, then the
>>>>> transaction is was aborted (by simply forgetting about it - TDB
>>>>> does not change the core databased as the application progress - it
>>>>> uses
>>>
>>> journalling.
>>>>>
>>>>>
>>>>> If the machine crashed after the commit() -- even if before end()
>>>>> -- the transaction committed and will be recovered on next startup.
>>>>>
>>>>> If the machine crashed during commit then the transaction either
>>>>> completely committed or not all (and will be aborted). There is no
>>>>> "half-committed". The true internal commit point is the sync of a
>>>>> single disk block.
>>>>>
>>>>> Your code may have to check whether the transaction happened or not
>>>>> - any transaction system has a point where a commit has been made
>>>>> but before the application is informed.
>>>>>
>>>>>
>>>>> When i relaunch the application what is the state of the
>>>>> transaction that
>>>>>>
>>>>>> wasn't ended (because of the crash) ?
>>>>>> Will i be able to start a new one with ds.begin ?
>>>>>>
>>>>>
>>>>> Yes.
>>>>>
>>>>> Andy
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
Re: Question about transactions
Posted by David Jordan <da...@bellsouth.net>.
Right, I forgot this morning about the effect of a long running read transaction.
Frederic, could you have a long running read transaction, causing the journal to continue to grow?
I guess the only way around this would be to have two journals, one with a block pre-image, another with the updates.
On May 13, 2013, at 4:30 PM, Andy Seaborne wrote:
> On 13/05/13 15:38, David Jordan wrote:
>> My understanding is that this may occur later, but better to let the experts say for certain.
>
> You are exactly right. When used directly, TDB may buffer 10 commits before flushign to disk but the app does not see this - the code manages to know whether to look in the main DB or the journal.
>
> In Fuseki, there is little benfit to this and it writes back fro the journal to teh DB when ver it can.
>
> However, it can only write back if there is no READ transaction in progress. Such a read-txn needs the old data to be around so the journal can't overwrite it.
>
> If there is a long lived transaction, the journal keeps growing.
>
> Andy
>
>>
>> -----Original Message-----
>> From: Frederic Toublanc [mailto:frederic.toublanc@telemis.com]
>> Sent: Monday, May 13, 2013 10:34 AM
>> To: users@jena.apache.org
>> Subject: Re: Question about transactions
>>
>> Ok i dont undestand.
>>
>> I though that ds.commit() really updated the database files...
>>
>>
>> 2013/5/13 David Jordan <Da...@sas.com>
>>
>>> I assume the journal file will shrink in size once the updates have
>>> actually been applied to the database files?
>>> If no transactions are in progress, is there a way to initiate this
>>> transfer of data from the journal to the database?
>>> It may also be of interest to the application to hold off this
>>> transfer from the journal to the database until a certain point in time.
>>>
>>> -----Original Message-----
>>> From: Frederic Toublanc [mailto:frederic.toublanc@telemis.com]
>>> Sent: Monday, May 13, 2013 10:08 AM
>>> To: users@jena.apache.org
>>> Subject: Re: Question about transactions
>>>
>>> Ok now i have an other problem.
>>> Each time i end i write transaction the size of the TDB keeps growing
>>> because of the file journal.jrnl.
>>>
>>>
>>> 2013/5/13 Frederic Toublanc <fr...@telemis.com>
>>>
>>>> Ok thx a lot :)
>>>>
>>>> Yes by "everythin" i meant that the JVM dies or a power failure.
>>>>
>>>>
>>>>
>>>> 2013/5/13 Andy Seaborne <an...@apache.org>
>>>>
>>>>> On 13/05/13 09:37, Frederic Toublanc wrote:
>>>>>
>>>>>> Hello everyone,
>>>>>>
>>>>>> I have a question concerning transactions.
>>>>>> I dont find any answer in the documentation so here is the questions.
>>>>>>
>>>>>> I start a transaction like this :
>>>>>>
>>>>>> ds.begin(ReadWrite.write);
>>>>>>
>>>>>> ds.getDefaultModel();
>>>>>>
>>>>>> try {
>>>>>>
>>>>>> ds.commit;
>>>>>> } catch (Exception ex) {
>>>>>>
>>>>>
>>>>> catch (Throwable ex) is slightly better.
>>>>>
>>>>> ds.abort();
>>>>>>
>>>>>
>>>>> } finally {
>>>>>> ds.end();}
>>>>>>
>>>>>> If in the transction everything crashes so the transction is still
>>>>>> active.
>>>>>>
>>>>>
>>>>> By "everything" you mean the JVM dies suddenly and unexpectedly?
>>>>> Power failure, whale falling on data centre (yes - it has
>>>>> happened!)
>>>>>
>>>>> TDB does assume fail-stop of hardware and OS - no ongoing random
>>>>> partial failures or silently corrupting RAM although it does
>>>>> checksum the transaction log entries.
>>>>>
>>>>> On restart transaction is either commited or not.
>>>>>
>>>>> If the machine crashed before the commit was called, then the
>>>>> transaction is was aborted (by simply forgetting about it - TDB
>>>>> does not change the core databased as the application progress - it
>>>>> uses
>>> journalling.
>>>>>
>>>>> If the machine crashed after the commit() -- even if before end()
>>>>> -- the transaction committed and will be recovered on next startup.
>>>>>
>>>>> If the machine crashed during commit then the transaction either
>>>>> completely committed or not all (and will be aborted). There is no
>>>>> "half-committed". The true internal commit point is the sync of a
>>>>> single disk block.
>>>>>
>>>>> Your code may have to check whether the transaction happened or not
>>>>> - any transaction system has a point where a commit has been made
>>>>> but before the application is informed.
>>>>>
>>>>>
>>>>> When i relaunch the application what is the state of the
>>>>> transaction that
>>>>>> wasn't ended (because of the crash) ?
>>>>>> Will i be able to start a new one with ds.begin ?
>>>>>>
>>>>>
>>>>> Yes.
>>>>>
>>>>> Andy
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
Re: Question about transactions
Posted by Andy Seaborne <an...@apache.org>.
On 13/05/13 15:38, David Jordan wrote:
> My understanding is that this may occur later, but better to let the experts say for certain.
You are exactly right. When used directly, TDB may buffer 10 commits
before flushign to disk but the app does not see this - the code manages
to know whether to look in the main DB or the journal.
In Fuseki, there is little benfit to this and it writes back fro the
journal to teh DB when ver it can.
However, it can only write back if there is no READ transaction in
progress. Such a read-txn needs the old data to be around so the
journal can't overwrite it.
If there is a long lived transaction, the journal keeps growing.
Andy
>
> -----Original Message-----
> From: Frederic Toublanc [mailto:frederic.toublanc@telemis.com]
> Sent: Monday, May 13, 2013 10:34 AM
> To: users@jena.apache.org
> Subject: Re: Question about transactions
>
> Ok i dont undestand.
>
> I though that ds.commit() really updated the database files...
>
>
> 2013/5/13 David Jordan <Da...@sas.com>
>
>> I assume the journal file will shrink in size once the updates have
>> actually been applied to the database files?
>> If no transactions are in progress, is there a way to initiate this
>> transfer of data from the journal to the database?
>> It may also be of interest to the application to hold off this
>> transfer from the journal to the database until a certain point in time.
>>
>> -----Original Message-----
>> From: Frederic Toublanc [mailto:frederic.toublanc@telemis.com]
>> Sent: Monday, May 13, 2013 10:08 AM
>> To: users@jena.apache.org
>> Subject: Re: Question about transactions
>>
>> Ok now i have an other problem.
>> Each time i end i write transaction the size of the TDB keeps growing
>> because of the file journal.jrnl.
>>
>>
>> 2013/5/13 Frederic Toublanc <fr...@telemis.com>
>>
>>> Ok thx a lot :)
>>>
>>> Yes by "everythin" i meant that the JVM dies or a power failure.
>>>
>>>
>>>
>>> 2013/5/13 Andy Seaborne <an...@apache.org>
>>>
>>>> On 13/05/13 09:37, Frederic Toublanc wrote:
>>>>
>>>>> Hello everyone,
>>>>>
>>>>> I have a question concerning transactions.
>>>>> I dont find any answer in the documentation so here is the questions.
>>>>>
>>>>> I start a transaction like this :
>>>>>
>>>>> ds.begin(ReadWrite.write);
>>>>>
>>>>> ds.getDefaultModel();
>>>>>
>>>>> try {
>>>>>
>>>>> ds.commit;
>>>>> } catch (Exception ex) {
>>>>>
>>>>
>>>> catch (Throwable ex) is slightly better.
>>>>
>>>> ds.abort();
>>>>>
>>>>
>>>> } finally {
>>>>> ds.end();}
>>>>>
>>>>> If in the transction everything crashes so the transction is still
>>>>> active.
>>>>>
>>>>
>>>> By "everything" you mean the JVM dies suddenly and unexpectedly?
>>>> Power failure, whale falling on data centre (yes - it has
>>>> happened!)
>>>>
>>>> TDB does assume fail-stop of hardware and OS - no ongoing random
>>>> partial failures or silently corrupting RAM although it does
>>>> checksum the transaction log entries.
>>>>
>>>> On restart transaction is either commited or not.
>>>>
>>>> If the machine crashed before the commit was called, then the
>>>> transaction is was aborted (by simply forgetting about it - TDB
>>>> does not change the core databased as the application progress - it
>>>> uses
>> journalling.
>>>>
>>>> If the machine crashed after the commit() -- even if before end()
>>>> -- the transaction committed and will be recovered on next startup.
>>>>
>>>> If the machine crashed during commit then the transaction either
>>>> completely committed or not all (and will be aborted). There is no
>>>> "half-committed". The true internal commit point is the sync of a
>>>> single disk block.
>>>>
>>>> Your code may have to check whether the transaction happened or not
>>>> - any transaction system has a point where a commit has been made
>>>> but before the application is informed.
>>>>
>>>>
>>>> When i relaunch the application what is the state of the
>>>> transaction that
>>>>> wasn't ended (because of the crash) ?
>>>>> Will i be able to start a new one with ds.begin ?
>>>>>
>>>>
>>>> Yes.
>>>>
>>>> Andy
>>>>
>>>>
>>>
>>
>>
>
RE: Question about transactions
Posted by David Jordan <Da...@sas.com>.
My understanding is that this may occur later, but better to let the experts say for certain.
-----Original Message-----
From: Frederic Toublanc [mailto:frederic.toublanc@telemis.com]
Sent: Monday, May 13, 2013 10:34 AM
To: users@jena.apache.org
Subject: Re: Question about transactions
Ok i dont undestand.
I though that ds.commit() really updated the database files...
2013/5/13 David Jordan <Da...@sas.com>
> I assume the journal file will shrink in size once the updates have
> actually been applied to the database files?
> If no transactions are in progress, is there a way to initiate this
> transfer of data from the journal to the database?
> It may also be of interest to the application to hold off this
> transfer from the journal to the database until a certain point in time.
>
> -----Original Message-----
> From: Frederic Toublanc [mailto:frederic.toublanc@telemis.com]
> Sent: Monday, May 13, 2013 10:08 AM
> To: users@jena.apache.org
> Subject: Re: Question about transactions
>
> Ok now i have an other problem.
> Each time i end i write transaction the size of the TDB keeps growing
> because of the file journal.jrnl.
>
>
> 2013/5/13 Frederic Toublanc <fr...@telemis.com>
>
> > Ok thx a lot :)
> >
> > Yes by "everythin" i meant that the JVM dies or a power failure.
> >
> >
> >
> > 2013/5/13 Andy Seaborne <an...@apache.org>
> >
> >> On 13/05/13 09:37, Frederic Toublanc wrote:
> >>
> >>> Hello everyone,
> >>>
> >>> I have a question concerning transactions.
> >>> I dont find any answer in the documentation so here is the questions.
> >>>
> >>> I start a transaction like this :
> >>>
> >>> ds.begin(ReadWrite.write);
> >>>
> >>> ds.getDefaultModel();
> >>>
> >>> try {
> >>>
> >>> ds.commit;
> >>> } catch (Exception ex) {
> >>>
> >>
> >> catch (Throwable ex) is slightly better.
> >>
> >> ds.abort();
> >>>
> >>
> >> } finally {
> >>> ds.end();}
> >>>
> >>> If in the transction everything crashes so the transction is still
> >>> active.
> >>>
> >>
> >> By "everything" you mean the JVM dies suddenly and unexpectedly?
> >> Power failure, whale falling on data centre (yes - it has
> >> happened!)
> >>
> >> TDB does assume fail-stop of hardware and OS - no ongoing random
> >> partial failures or silently corrupting RAM although it does
> >> checksum the transaction log entries.
> >>
> >> On restart transaction is either commited or not.
> >>
> >> If the machine crashed before the commit was called, then the
> >> transaction is was aborted (by simply forgetting about it - TDB
> >> does not change the core databased as the application progress - it
> >> uses
> journalling.
> >>
> >> If the machine crashed after the commit() -- even if before end()
> >> -- the transaction committed and will be recovered on next startup.
> >>
> >> If the machine crashed during commit then the transaction either
> >> completely committed or not all (and will be aborted). There is no
> >> "half-committed". The true internal commit point is the sync of a
> >> single disk block.
> >>
> >> Your code may have to check whether the transaction happened or not
> >> - any transaction system has a point where a commit has been made
> >> but before the application is informed.
> >>
> >>
> >> When i relaunch the application what is the state of the
> >> transaction that
> >>> wasn't ended (because of the crash) ?
> >>> Will i be able to start a new one with ds.begin ?
> >>>
> >>
> >> Yes.
> >>
> >> Andy
> >>
> >>
> >
>
>
Re: Question about transactions
Posted by Frederic Toublanc <fr...@telemis.com>.
Ok i dont undestand.
I though that ds.commit() really updated the database files...
2013/5/13 David Jordan <Da...@sas.com>
> I assume the journal file will shrink in size once the updates have
> actually been applied to the database files?
> If no transactions are in progress, is there a way to initiate this
> transfer of data from the journal to the database?
> It may also be of interest to the application to hold off this transfer
> from the journal to the database until a certain point in time.
>
> -----Original Message-----
> From: Frederic Toublanc [mailto:frederic.toublanc@telemis.com]
> Sent: Monday, May 13, 2013 10:08 AM
> To: users@jena.apache.org
> Subject: Re: Question about transactions
>
> Ok now i have an other problem.
> Each time i end i write transaction the size of the TDB keeps growing
> because of the file journal.jrnl.
>
>
> 2013/5/13 Frederic Toublanc <fr...@telemis.com>
>
> > Ok thx a lot :)
> >
> > Yes by "everythin" i meant that the JVM dies or a power failure.
> >
> >
> >
> > 2013/5/13 Andy Seaborne <an...@apache.org>
> >
> >> On 13/05/13 09:37, Frederic Toublanc wrote:
> >>
> >>> Hello everyone,
> >>>
> >>> I have a question concerning transactions.
> >>> I dont find any answer in the documentation so here is the questions.
> >>>
> >>> I start a transaction like this :
> >>>
> >>> ds.begin(ReadWrite.write);
> >>>
> >>> ds.getDefaultModel();
> >>>
> >>> try {
> >>>
> >>> ds.commit;
> >>> } catch (Exception ex) {
> >>>
> >>
> >> catch (Throwable ex) is slightly better.
> >>
> >> ds.abort();
> >>>
> >>
> >> } finally {
> >>> ds.end();}
> >>>
> >>> If in the transction everything crashes so the transction is still
> >>> active.
> >>>
> >>
> >> By "everything" you mean the JVM dies suddenly and unexpectedly?
> >> Power failure, whale falling on data centre (yes - it has happened!)
> >>
> >> TDB does assume fail-stop of hardware and OS - no ongoing random
> >> partial failures or silently corrupting RAM although it does checksum
> >> the transaction log entries.
> >>
> >> On restart transaction is either commited or not.
> >>
> >> If the machine crashed before the commit was called, then the
> >> transaction is was aborted (by simply forgetting about it - TDB does
> >> not change the core databased as the application progress - it uses
> journalling.
> >>
> >> If the machine crashed after the commit() -- even if before end() --
> >> the transaction committed and will be recovered on next startup.
> >>
> >> If the machine crashed during commit then the transaction either
> >> completely committed or not all (and will be aborted). There is no
> >> "half-committed". The true internal commit point is the sync of a
> >> single disk block.
> >>
> >> Your code may have to check whether the transaction happened or not -
> >> any transaction system has a point where a commit has been made but
> >> before the application is informed.
> >>
> >>
> >> When i relaunch the application what is the state of the transaction
> >> that
> >>> wasn't ended (because of the crash) ?
> >>> Will i be able to start a new one with ds.begin ?
> >>>
> >>
> >> Yes.
> >>
> >> Andy
> >>
> >>
> >
>
>
RE: Question about transactions
Posted by David Jordan <Da...@sas.com>.
I assume the journal file will shrink in size once the updates have actually been applied to the database files?
If no transactions are in progress, is there a way to initiate this transfer of data from the journal to the database?
It may also be of interest to the application to hold off this transfer from the journal to the database until a certain point in time.
-----Original Message-----
From: Frederic Toublanc [mailto:frederic.toublanc@telemis.com]
Sent: Monday, May 13, 2013 10:08 AM
To: users@jena.apache.org
Subject: Re: Question about transactions
Ok now i have an other problem.
Each time i end i write transaction the size of the TDB keeps growing because of the file journal.jrnl.
2013/5/13 Frederic Toublanc <fr...@telemis.com>
> Ok thx a lot :)
>
> Yes by "everythin" i meant that the JVM dies or a power failure.
>
>
>
> 2013/5/13 Andy Seaborne <an...@apache.org>
>
>> On 13/05/13 09:37, Frederic Toublanc wrote:
>>
>>> Hello everyone,
>>>
>>> I have a question concerning transactions.
>>> I dont find any answer in the documentation so here is the questions.
>>>
>>> I start a transaction like this :
>>>
>>> ds.begin(ReadWrite.write);
>>>
>>> ds.getDefaultModel();
>>>
>>> try {
>>>
>>> ds.commit;
>>> } catch (Exception ex) {
>>>
>>
>> catch (Throwable ex) is slightly better.
>>
>> ds.abort();
>>>
>>
>> } finally {
>>> ds.end();}
>>>
>>> If in the transction everything crashes so the transction is still
>>> active.
>>>
>>
>> By "everything" you mean the JVM dies suddenly and unexpectedly?
>> Power failure, whale falling on data centre (yes - it has happened!)
>>
>> TDB does assume fail-stop of hardware and OS - no ongoing random
>> partial failures or silently corrupting RAM although it does checksum
>> the transaction log entries.
>>
>> On restart transaction is either commited or not.
>>
>> If the machine crashed before the commit was called, then the
>> transaction is was aborted (by simply forgetting about it - TDB does
>> not change the core databased as the application progress - it uses journalling.
>>
>> If the machine crashed after the commit() -- even if before end() --
>> the transaction committed and will be recovered on next startup.
>>
>> If the machine crashed during commit then the transaction either
>> completely committed or not all (and will be aborted). There is no
>> "half-committed". The true internal commit point is the sync of a
>> single disk block.
>>
>> Your code may have to check whether the transaction happened or not -
>> any transaction system has a point where a commit has been made but
>> before the application is informed.
>>
>>
>> When i relaunch the application what is the state of the transaction
>> that
>>> wasn't ended (because of the crash) ?
>>> Will i be able to start a new one with ds.begin ?
>>>
>>
>> Yes.
>>
>> Andy
>>
>>
>
Re: Question about transactions
Posted by Frederic Toublanc <fr...@telemis.com>.
Ok now i have an other problem.
Each time i end i write transaction the size of the TDB keeps growing
because of the file journal.jrnl.
2013/5/13 Frederic Toublanc <fr...@telemis.com>
> Ok thx a lot :)
>
> Yes by "everythin" i meant that the JVM dies or a power failure.
>
>
>
> 2013/5/13 Andy Seaborne <an...@apache.org>
>
>> On 13/05/13 09:37, Frederic Toublanc wrote:
>>
>>> Hello everyone,
>>>
>>> I have a question concerning transactions.
>>> I dont find any answer in the documentation so here is the questions.
>>>
>>> I start a transaction like this :
>>>
>>> ds.begin(ReadWrite.write);
>>>
>>> ds.getDefaultModel();
>>>
>>> try {
>>>
>>> ds.commit;
>>> } catch (Exception ex) {
>>>
>>
>> catch (Throwable ex) is slightly better.
>>
>> ds.abort();
>>>
>>
>> } finally {
>>> ds.end();}
>>>
>>> If in the transction everything crashes so the transction is still
>>> active.
>>>
>>
>> By "everything" you mean the JVM dies suddenly and unexpectedly? Power
>> failure, whale falling on data centre (yes - it has happened!)
>>
>> TDB does assume fail-stop of hardware and OS - no ongoing random partial
>> failures or silently corrupting RAM although it does checksum the
>> transaction log entries.
>>
>> On restart transaction is either commited or not.
>>
>> If the machine crashed before the commit was called, then the transaction
>> is was aborted (by simply forgetting about it - TDB does not change the
>> core databased as the application progress - it uses journalling.
>>
>> If the machine crashed after the commit() -- even if before end() -- the
>> transaction committed and will be recovered on next startup.
>>
>> If the machine crashed during commit then the transaction either
>> completely committed or not all (and will be aborted). There is no
>> "half-committed". The true internal commit point is the sync of a single
>> disk block.
>>
>> Your code may have to check whether the transaction happened or not - any
>> transaction system has a point where a commit has been made but before the
>> application is informed.
>>
>>
>> When i relaunch the application what is the state of the transaction that
>>> wasn't ended (because of the crash) ?
>>> Will i be able to start a new one with ds.begin ?
>>>
>>
>> Yes.
>>
>> Andy
>>
>>
>
Re: Question about transactions
Posted by Frederic Toublanc <fr...@telemis.com>.
Ok thx a lot :)
Yes by "everythin" i meant that the JVM dies or a power failure.
2013/5/13 Andy Seaborne <an...@apache.org>
> On 13/05/13 09:37, Frederic Toublanc wrote:
>
>> Hello everyone,
>>
>> I have a question concerning transactions.
>> I dont find any answer in the documentation so here is the questions.
>>
>> I start a transaction like this :
>>
>> ds.begin(ReadWrite.write);
>>
>> ds.getDefaultModel();
>>
>> try {
>>
>> ds.commit;
>> } catch (Exception ex) {
>>
>
> catch (Throwable ex) is slightly better.
>
> ds.abort();
>>
>
> } finally {
>> ds.end();}
>>
>> If in the transction everything crashes so the transction is still active.
>>
>
> By "everything" you mean the JVM dies suddenly and unexpectedly? Power
> failure, whale falling on data centre (yes - it has happened!)
>
> TDB does assume fail-stop of hardware and OS - no ongoing random partial
> failures or silently corrupting RAM although it does checksum the
> transaction log entries.
>
> On restart transaction is either commited or not.
>
> If the machine crashed before the commit was called, then the transaction
> is was aborted (by simply forgetting about it - TDB does not change the
> core databased as the application progress - it uses journalling.
>
> If the machine crashed after the commit() -- even if before end() -- the
> transaction committed and will be recovered on next startup.
>
> If the machine crashed during commit then the transaction either
> completely committed or not all (and will be aborted). There is no
> "half-committed". The true internal commit point is the sync of a single
> disk block.
>
> Your code may have to check whether the transaction happened or not - any
> transaction system has a point where a commit has been made but before the
> application is informed.
>
>
> When i relaunch the application what is the state of the transaction that
>> wasn't ended (because of the crash) ?
>> Will i be able to start a new one with ds.begin ?
>>
>
> Yes.
>
> Andy
>
>
Re: Question about transactions
Posted by Andy Seaborne <an...@apache.org>.
On 13/05/13 09:37, Frederic Toublanc wrote:
> Hello everyone,
>
> I have a question concerning transactions.
> I dont find any answer in the documentation so here is the questions.
>
> I start a transaction like this :
>
> ds.begin(ReadWrite.write);
>
> ds.getDefaultModel();
>
> try {
>
> ds.commit;
> } catch (Exception ex) {
catch (Throwable ex) is slightly better.
> ds.abort();
> } finally {
> ds.end();}
>
> If in the transction everything crashes so the transction is still active.
By "everything" you mean the JVM dies suddenly and unexpectedly? Power
failure, whale falling on data centre (yes - it has happened!)
TDB does assume fail-stop of hardware and OS - no ongoing random partial
failures or silently corrupting RAM although it does checksum the
transaction log entries.
On restart transaction is either commited or not.
If the machine crashed before the commit was called, then the
transaction is was aborted (by simply forgetting about it - TDB does not
change the core databased as the application progress - it uses journalling.
If the machine crashed after the commit() -- even if before end() -- the
transaction committed and will be recovered on next startup.
If the machine crashed during commit then the transaction either
completely committed or not all (and will be aborted). There is no
"half-committed". The true internal commit point is the sync of a
single disk block.
Your code may have to check whether the transaction happened or not -
any transaction system has a point where a commit has been made but
before the application is informed.
> When i relaunch the application what is the state of the transaction that
> wasn't ended (because of the crash) ?
> Will i be able to start a new one with ds.begin ?
Yes.
Andy