Disk space requirement for building on Windows (was: Re: [jira] [Commented] (JENA-897) jena-jdbc-tdb tests use %TEMP% instead of target/)

[Stian Soiland-Reyes <st...@apache.org>]

 Thanks, Rob!

I tried looking yesterday at ways to reduce the disk space
requirements when building on Windows - including truncating the files
after closing. This seems to require deep changes into TDBs
ChannelManager which keeps the corresponding FileChannels - perhaps a
new method for that purpose?

https://github.com/apache/jena/blob/master/jena-tdb/src/main/java/com/hp/hpl/jena/tdb/base/file/ChannelManager.java


It seems on Windows with Oracle/OpenJDK you can call System.gc() to
(hopefully) release the ByteBuffers that lock the memory regions (and
then making the files deletable) - but this adds a significant
overhead. The dispose methods on ByteBufferImpls are not easily
accessible - you would need some introspection hackery to get hold of
that cleaner() and that would of course only work on Oracle/OpenJDK.
as fc.map() still does the same thing.

Close your eyes -  GPL3!
https://github.com/stain/jdk8u/blob/master/src/share/classes/java/nio/Direct-X-Buffer.java.template#L72
http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/java/nio/DirectByteBuffer.java/#72


I tried using the FileChannels from JDK7 NIO2 (e.g.
FileChannel.open(Path)) instead of through RandomAccessFile - but it
did not make any difference


Perhaps System.gc() is not worth it in general (* on Windows) when
closing a dataset - I tried to modify the ChannelManager to always do
this on release, it meant each test in jena-jdbc-tdb took 1.5s instead
of 0.2s, but it did allow me to delete the used folders from target/
while the JVM/test was running.

For the tests we could do something like for every 10 tests do
System.gc() and wipe the old data.

Perhaps Fuseki 2 could do System.gc() on [Remove] && SystemTDB.isWindows.



On 10 March 2015 at 10:00, ASF GitHub Bot (JIRA) <ji...@apache.org> wrote:
>
>     [ https://issues.apache.org/jira/browse/JENA-897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14354617#comment-14354617 ]
>
> ASF GitHub Bot commented on JENA-897:
> -------------------------------------
>
> Github user asfgit closed the pull request at:
>
>     https://github.com/apache/jena/pull/41
>
>
>> jena-jdbc-tdb tests use %TEMP% instead of target/
>> -------------------------------------------------
>>
>>                 Key: JENA-897
>>                 URL: https://issues.apache.org/jira/browse/JENA-897
>>             Project: Apache Jena
>>          Issue Type: Bug
>>          Components: JDBC
>>    Affects Versions: Jena 2.12.1, Jena 2.13.0
>>         Environment: Windowx 8.0 x64, C: with 34 GB free
>>            Reporter: Stian Soiland-Reyes
>>            Priority: Critical
>>             Fix For: Jena 2.13.1
>>
>>
>> .. and thus mvn clean install on Windows will easily consume 37 GB on C: and run out of disk space - even if Jena is built on a larger partition.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)



-- 
Stian Soiland-Reyes
Apache Taverna (incubating), Apache Commons RDF (incubating)
http://orcid.org/0000-0001-9842-9718