Re: Jena TDB errors : how to debug ?

[Jean-Marc Vanel <je...@gmail.com>]

2014-02-21 18:03 GMT+01:00 Andy Seaborne <an...@apache.org>:

> Hi there,
>
> Which version is this?
> What OS are you running and what filing system for the DB?


Sorry, I should have said that at once,
and sorry for the delay in answering.

Ubuntu 13.10 with all updates, and default file system , ext4 I think.

On 21/02/14 10:28, Jean-Marc Vanel wrote:
>
>> I'm encountering every now and then TDBTransactionException's. These
>> errors
>> may be due to my code not respecting the transaction gotchas, but also to
>> real bugs. These errors may be due to my code not respecting the
>> transaction gotchas, but also to real bugs.
>>
>> The context : I'm developing Swing applications
>> (DataGUI<http://svn.code.sf.net/p/eulergui/code/trunk/
>> eulergui/html/documentation.html#Data1>
>> , DomainApplication<http://svn.code.sf.net/p/eulergui/code/
>> trunk/eulergui/html/documentation.html#L7253>
>> ) that accumulate documents from the web in a TDB store that is used as a
>>
>> cache.
>>
>> The writings occur in threads (Scala futures) associated to the URI
>> downloaded, and to the named graph.
>>
>> Also, the user can add triples (in a special named graph) that are at once
>> commited.
>>
>> So, there is plenty of room for errors, plus the application is sometimes
>> killed.
>>
>
> That might explain one occurrence of the exceptions - the bad checksum.
>  If so, it's safe - the transaction was not truly committed and the app
> wasn't told it was.


You mean the app  wasn't told it was commited .
( at first I understood :
the app  wasn't told it was NOT commited )


>
>  Here are examples of errors:
>>
>
> Do you longer stacktraces for each one?

 No, but I have several others that I'll send today.

>
>
>> [ERROR] - 2014-02-21 09:54:52,181 [ForkJoinPool-1-worker-1] TDB
>> (JournalControl.java:replay:259)
>>
>> Exception during journal replay
>>
>>
> The journal couldn't be replayed because the last entry was messed up but
> that's not surprising if ...
>
>
>
>  com.hp.hpl.jena.tdb.transaction.TDBTransactionException: Checksum
>> error reading from the Journal.
>>
>
> (ditto)
>
> Exception after commit point : transaction commited but internal
>> status not recorded properly
>>
>
==> This error message is misleading from what you write just below;
maybe it should say "Exception after commit point : attempt
to commit transaction, but internal
status not recorded properly"

I seems that there is sometimes a semantic confusion between transaction
from the service provider point of view, and the application  point of view.

>
> [commit point is the internal commit - the journal has been sync'ed]
>
> An attempt to commit failed during writing the transaction log - there
> should be an exception stacktrace.
>
> The transaction did not commit properly from the applications point of
> view and it all depends on the journal.  The journal is the ultimate
> arbiter of what committed and what didn't based on recovery.


That means, that, although from  application point of view it has been
reported as aborted, there may well be some data bits recoverable after
database restart ?


>
>
>> com.hp.hpl.jena.tdb.transaction.TDBTransactionException: Checksum
>> error reading from the Journal.
>>
>
> Something else - a way to cause this is to exit the JVM forcefully (while
> a journal entry is being written).
>
>
>  I need some advice about debugging, like displaying dataset, graph,
>> journal
>> status, or adding assertions, or whatever is possible. I'm willing to
>> contribute to make TDB more robust by providing repeatable situations.
>>
>> Another question is : how to repair a broken TDB ?
>>
>
> Attempt to start up TDB once, ignore any errors - this commits any pending
> but complete transactions in the journal.  The last one about Checksum
> error reading should be the last transaction and it's not really committed
> from an external recovery point of view.
>
> Exit, then delete the journal file.


Is this what tdbrecovery does ?
The help is not explicit :
 java -cp $JAR tdb.tdbrecovery --help
neither is :
java -cp $JAR tdb.tdbclean --help

Restart.  Take a separate backup.
>
>
>  Or maybe the question is : can a TDB be broken at all, or is it my bad
>> application code that repeatedly creates similar problems?
>>
>
> If you run two copies of your code at the same time, then you will break
> the database.
>
===>
 This is likely to happen, since this is a GUI application;
I'll have to prevent several concurrent starts.

is your environment doing any classlaoder games?  If so, it might have two
> copies of the TDB engine running which is similar recipe for disaster (two
> copies of the statics that manage the database state).
>
NO.

>
>
>> Kind regards
>>
>>
>         Andy
>



-- 
Jean-Marc Vanel
Déductions SARL - Consulting, services, training,
Rule-based programming, Semantic Web
http://deductions-software.com/
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui

Re: Jena TDB errors : how to debug ?

[Andy Seaborne <an...@apache.org>]

On 03/03/14 11:35, Jean-Marc Vanel wrote:
> 2014-02-21 18:03 GMT+01:00 Andy Seaborne <an...@apache.org>:
...
>>   I need some advice about debugging, like displaying dataset, graph,
>>> journal
>>> status, or adding assertions, or whatever is possible. I'm willing to
>>> contribute to make TDB more robust by providing repeatable situations.
>>>
>>> Another question is : how to repair a broken TDB ?
>>>
>>
>> Attempt to start up TDB once, ignore any errors - this commits any pending
>> but complete transactions in the journal.  The last one about Checksum
>> error reading should be the last transaction and it's not really committed
>> from an external recovery point of view.
>>
>> Exit, then delete the journal file.
>
>
> Is this what tdbrecovery does ?
> The help is not explicit :
>   java -cp $JAR tdb.tdbrecovery --help

yes - just runs journal recovery (that run anyway everytime you start TDB).

> neither is :
> java -cp $JAR tdb.tdbclean --help

no - that deletes files.

>
> Restart.  Take a separate backup.
>>
>>
>>   Or maybe the question is : can a TDB be broken at all, or is it my bad
>>> application code that repeatedly creates similar problems?
>>>
>>
>> If you run two copies of your code at the same time, then you will break
>> the database.
>>
> ===>
>   This is likely to happen, since this is a GUI application;
> I'll have to prevent several concurrent starts.

Yes - you'll have to make sure only one JVM is using a given location at 
the same time.

If you want to share a copy of TDB database between different JVMs, the 
thing to do is to use Fuseki as a "database server" - that shares the 
database safely across different clients.

	Andy