[jira] [Commented] (JENA-1379) Replace TDB NodeTableTrans

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/JENA-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115436#comment-16115436 ] 

ASF GitHub Bot commented on JENA-1379:
--------------------------------------

GitHub user afs opened a pull request:

    https://github.com/apache/jena/pull/272

    JENA-1379: Better (simpler, more robust) transactional NodeTables

    See [JENA-1379](https://issues.apache.org/jira/browse/JENA-1379) for more details.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/afs/jena tdb-nodetable-txn

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/jena/pull/272.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #272
    
----
commit e6e1b16aaca2c433120d61f2d7ad4edaaa1e22cf
Author: Andy Seaborne <an...@apache.org>
Date:   2017-08-04T16:14:33Z

    Build from ObjectFiles and BlockMgrs.
    
    Remove NodeTableBuilder
    Remove NodeTableTrans

----


> Replace TDB NodeTableTrans
> --------------------------
>
>                 Key: JENA-1379
>                 URL: https://issues.apache.org/jira/browse/JENA-1379
>             Project: Apache Jena
>          Issue Type: Bug
>          Components: TDB
>    Affects Versions: Jena 3.4.0
>            Reporter: Andy Seaborne
>            Assignee: Andy Seaborne
>
> TDB {{NodeTableTrans}} is complicated. It combines an existing {{NodeTable}} with an additional index (often in-memory) and a journal-like {{ObjectFile}} to hold new nodes added in a transaction. It has to maintain a mapping between the new nodes in the journal-ObjectFile and the eventual location on the main node file. On commit, it writes the journal-ObjectFile nodes to underlying index. There is a problem that writing the index isn't done completely safely. The window of vulnerability is quite small though (coordinating the index update and the object file update).
> {{NodeTableBuilder}} is part of the way TDB datasets get built. A simpler design is to make {{NodeTable}}s be built from the basic components on `BlockMgr`s and `ObjectFile`s (the two units of storage in TDB) in a fixed fashion. The potential flexibility of the current design has never been exploited.
> There are two parts to this change: they are independent.
> # a transactional index (based on the same machinery as the tuple indexes) and directly appending to the object file of the {{NodeTable}}.
> # independent transactional object file.
> Directly appending is safe because these files only grow. Only nodes in the associated index are accessible.  Abort resets the append point; a crash during a write transaction can, at worst, create unused junk in the object file but this is a trade-off of speed and recovery. A journalled addition object file would avoid junk in some crash situations, though it imposes a copy cost. It is proposed to go for simple+speed. "Simpler" is easier to make crash-safe.
> The alternative here is not to keep the existing code - there is some unused (and hence no deployment-tested) code in {{ObjectFileTransComplex}} (working name) for a more complicated journalled object file.
> The on-disk format is not changed except that existing (up to Jena 3.4.0) "dat-jrnl" files do not exist. Presence of indicates crash recovery is needed. The safest way is to require that recovery is done with the same version of TDB with a test in new code that notices and exist if it encounters old files. Oddly, old code should recover new version datasets correctly! All the work has been moved the the main index journal.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)