[jira] [Commented] (OAK-5326) Not able to move segment store directory on filesystem after closing FileStore

["Arek Kita (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15817643#comment-15817643 ] 

Arek Kita commented on OAK-5326:
--------------------------------

[~tomek.rekawek] I was able to reproduce the issue on a very simple code snippet using {{segment-tar}} file store builder on the {{srcPath}} that stores repository in {{segment}} format. The code normally throws {{InvalidFileStoreVersionException}} in such condition as the old repository is opened without migration by a newer {{segment-tar}} code.

Together with [~frm] we found that in case of exception {{close()}} cannot be called as it operates on variable that is newer returned and the code immediately throws an exception (code is presented below) which leads to leak (as Oak is not cleaning the repository state in such condition).

On Windows such directory cannot be moved and it is completely not related to MMAP (as: {{.withMemoryMapping(false)}} is used)

{code}
        fileStoreBuilder(new File(srcPath, OakUtils.SEGMENT_STORE_DIR))
                .withMaxFileSize(256)
                .withMemoryMapping(false)
                .buildReadOnly()
                .close();
{code}

[~frm] found 1 place where we might leak some file descriptors and the effort is reflected in OAK-5436 (leaking manifest opened file).
However during move operation different file seems to be locked (it is: {{journal.log}}, so the work to fix this is reflected in OAK-5438 (leaking journal.log file descriptor).

As both issues OAK-5436 and OAK-5438  are fixed, I'm resolving this issue too as everything seems to be working fine.
Thanks [~frm] for fixing!

> Not able to move segment store directory on filesystem after closing FileStore
> ------------------------------------------------------------------------------
>
>                 Key: OAK-5326
>                 URL: https://issues.apache.org/jira/browse/OAK-5326
>             Project: Jackrabbit Oak
>          Issue Type: Bug
>          Components: upgrade
>    Affects Versions: 1.5.13
>         Environment: Windows 2012 R2,
> Java(TM) SE Run-time Environment (build 1.8.0_74-b02)
>            Reporter: Arek Kita
>            Assignee: Arek Kita
>             Fix For: 1.5.18, 1.6
>
>
> The JVM process that uses {{oak-upgrade}} library for upgrading Oak repository is not able to move source (and probably destination) repository after making sure that both file stores are closed. 
> {code:title=oak-upgrade}
> 16.12.2016 17:15:31.463 INFO   o.a.j.o.u.RepositorySidegrade: Copying node #770000: 
> 16.12.2016 17:15:38.604 INFO   o.a.j.o.u.RepositorySidegrade: Copying node #780000: 
> 16.12.2016 17:15:39.495 INFO   o.a.j.o.u.RepositorySidegrade: Copying node #790000: 
> 16.12.2016 17:15:41.182 INFO   o.a.j.o.s.f.FileStore: TarMK closed: C:\workspace\repository-segment-tar-20161216-171300\segmentstore
> 16.12.2016 17:15:41.370 INFO   o.a.j.o.p.s.f.FileStore: TarMK closed: C:\workspace\repository\segmentstore
> 16.12.2016 17:17:11.604 ERROR  java.io.IOException: Unable to delete file: C:\workspace\repository\segmentstore\journal.log
> 	at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2381) ~
> 	at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1679) ~
> 	at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1575) ~
> 	at org.apache.commons.io.FileUtils.moveDirectory(FileUtils.java:2916) ~
> ...
> {code}
> Please also note that {{--disable-mmap}} option is passed to avoid OAK-4274.
> Is there any way to make all operations synchronised and finished after the migration is executed using the API method: {{OakUpgrade.migrate(MigrationOptions options, StoreArguments stores, DatastoreArguments datastores) throws IOException, CliArgumentException}} 
> This leads to a weird experience when filesystem clean up (reordering) is needed.
> On Unix the {{segmentstore}} folder might be moved as UNIX fs implementation is not so restrictive. On Windows it is *not possible*.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)