You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Roshan Naik <ro...@hortonworks.com> on 2013/05/30 06:37:16 UTC
Review Request: Fix for TestFlumeEventQueue failures in FileChannel on
Windows
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11536/
-----------------------------------------------------------
Review request for Flume.
Description
-------
Windows does not allow deletion of files that are in use. Java does not yet allow explicitly unmapping memory mapped files so until these files are unmapped by the GC, we cannot delete them explicitly.
Checkpoint files are being reused across tests. Deletion of checkpoint files fails in these tests as the file memory mapped.
Fix: Not reusing checkpoint files to avoid deletion problems. Also ensuring files are setup to be deleted by jvm on exit.
This addresses bug FLUME-2058.
https://issues.apache.org/jira/browse/FLUME-2058
Diffs
-----
flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFlumeEventQueue.java 203cbf2
Diff: https://reviews.apache.org/r/11536/diff/
Testing
-------
TestFlumeEventQueue test now pass on Windows.
Thanks,
Roshan Naik
Re: Review Request: Fix for TestFlumeEventQueue failures in FileChannel on
Windows
Posted by Hari Shreedharan <hs...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11536/#review22085
-----------------------------------------------------------
Roshan,
The idea behind this patch is good - but I'd like to keep the files in the tests named the same as the actual files being used by the File Channel. So I'd recommend creating the files as before itself, but in different temporary directories and setting all of them up for deletion at JVM exit.
- Hari Shreedharan
On May 30, 2013, 4:37 a.m., Roshan Naik wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11536/
> -----------------------------------------------------------
>
> (Updated May 30, 2013, 4:37 a.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> Windows does not allow deletion of files that are in use. Java does not yet allow explicitly unmapping memory mapped files so until these files are unmapped by the GC, we cannot delete them explicitly.
>
> Checkpoint files are being reused across tests. Deletion of checkpoint files fails in these tests as the file memory mapped.
>
> Fix: Not reusing checkpoint files to avoid deletion problems. Also ensuring files are setup to be deleted by jvm on exit.
>
>
> This addresses bug FLUME-2058.
> https://issues.apache.org/jira/browse/FLUME-2058
>
>
> Diffs
> -----
>
> flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFlumeEventQueue.java 203cbf2
>
> Diff: https://reviews.apache.org/r/11536/diff/
>
>
> Testing
> -------
>
> TestFlumeEventQueue test now pass on Windows.
>
>
> Thanks,
>
> Roshan Naik
>
>
Re: Review Request: Fix for TestFlumeEventQueue failures in FileChannel on
Windows
Posted by Mike Percy <mp...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11536/#review22274
-----------------------------------------------------------
I'm shooting for tomorrow for the first RC, if we can get this committed before then then that would be great
- Mike Percy
On May 30, 2013, 4:37 a.m., Roshan Naik wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11536/
> -----------------------------------------------------------
>
> (Updated May 30, 2013, 4:37 a.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> Windows does not allow deletion of files that are in use. Java does not yet allow explicitly unmapping memory mapped files so until these files are unmapped by the GC, we cannot delete them explicitly.
>
> Checkpoint files are being reused across tests. Deletion of checkpoint files fails in these tests as the file memory mapped.
>
> Fix: Not reusing checkpoint files to avoid deletion problems. Also ensuring files are setup to be deleted by jvm on exit.
>
>
> This addresses bug FLUME-2058.
> https://issues.apache.org/jira/browse/FLUME-2058
>
>
> Diffs
> -----
>
> flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFlumeEventQueue.java 203cbf2
>
> Diff: https://reviews.apache.org/r/11536/diff/
>
>
> Testing
> -------
>
> TestFlumeEventQueue test now pass on Windows.
>
>
> Thanks,
>
> Roshan Naik
>
>
Re: Review Request 11536: Fix for TestFlumeEventQueue failures in
FileChannel on Windows
Posted by Alexander Alten-Lorenz <al...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11536/#review23294
-----------------------------------------------------------
How's about using the flag FILE_SHARE_DELETE - but will be only supported in J7, unfortunately (http://www.docjar.com/html/api/sun/nio/fs/WindowsFileAttributeViews.java.html)?
- Alexander Alten-Lorenz
On May 30, 2013, 4:37 a.m., Roshan Naik wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11536/
> -----------------------------------------------------------
>
> (Updated May 30, 2013, 4:37 a.m.)
>
>
> Review request for Flume.
>
>
> Bugs: FLUME-2058
> https://issues.apache.org/jira/browse/FLUME-2058
>
>
> Repository: flume-git
>
>
> Description
> -------
>
> Windows does not allow deletion of files that are in use. Java does not yet allow explicitly unmapping memory mapped files so until these files are unmapped by the GC, we cannot delete them explicitly.
>
> Checkpoint files are being reused across tests. Deletion of checkpoint files fails in these tests as the file memory mapped.
>
> Fix: Not reusing checkpoint files to avoid deletion problems. Also ensuring files are setup to be deleted by jvm on exit.
>
>
> Diffs
> -----
>
> flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFlumeEventQueue.java 203cbf2
>
> Diff: https://reviews.apache.org/r/11536/diff/
>
>
> Testing
> -------
>
> TestFlumeEventQueue test now pass on Windows.
>
>
> Thanks,
>
> Roshan Naik
>
>