[jira] [Resolved] (JENA-91) extremely large buffer is being created in ObjectFileStorage

["Andy Seaborne (Resolved) (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/JENA-91?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andy Seaborne resolved JENA-91.
-------------------------------

    Resolution: Fixed

Fixes for the original situation (large buffer - corrupted nodes.dat file) have been applied to the TxTDB codebase, tests added and they pass on Linux and Windows, in mapped and direct mode.  (Windows tests for mapped mode and deleting databases can not pass due to the Sun JVM bug.)

Build: TxTDB development build 20111006.082548-13 or later
http://openjena.org/repo-dev/com/hp/hpl/jena/tx-tdb/0.9.0-SNAPSHOT/

or 
https://repository.apache.org/content/repositories/snapshots/org/apache/jena/jena-tdb/

(the latter is the experimental Apache build - same code, different build process)

This JIRA has picked up several different problems. To be clear if/when issues arise, then rather than reopening it, please open a new JIRA specific to the report, 
attach test material to the new JIRA and relate it to this one.

                
> extremely large buffer is being created in ObjectFileStorage
> ------------------------------------------------------------
>
>                 Key: JENA-91
>                 URL: https://issues.apache.org/jira/browse/JENA-91
>             Project: Jena
>          Issue Type: Bug
>          Components: TDB
>            Reporter: Simon Helsen
>            Assignee: Andy Seaborne
>            Priority: Critical
>         Attachments: JENA-91_NodeTableTrans_r1159121.patch, TestTransSystem.patch, TestTransSystem2.patch, TestTransSystem3.patch, TestTransSystem4.patch
>
>
> I tried to debug the OME and check why a bytebuffer is causing my native memory to explode in almost no time. It all seems to happen in this bit of code in com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage (lines 243 onwards)
>   // No - it's in the underlying file storage.
>         lengthBuffer.clear() ;
>         int x = file.read(lengthBuffer, loc) ;
>         if ( x != 4 )
>             throw new FileException("ObjectFile.read("+loc+")["+filesize+"]["+file.size()+"]: Failed to read the length : got "+x+" bytes") ;
>         int len = lengthBuffer.getInt(0) ;
>         ByteBuffer bb = ByteBuffer.allocate(len) ;
> My debugger shows that x==4. It also shows the lengthBuffer has the following content: [111, 110, 61, 95]. This amounts to the value of len=1869495647, which is rather a lot :-) Obviously, the next statement (ByteBuffer.allocate) causes the OME.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira