kafka pull request: Unmap before resizing

[lizziew <gi...@git.apache.org>]

GitHub user lizziew opened a pull request:

    https://github.com/apache/kafka/pull/6

    Unmap before resizing

    While I was studying how MappedByteBuffer works, I saw a sharing runtime exception on Windows. I applied what I learned to generate a patch which uses an internal open JDK API to solve this problem.
    
    ---
    
    Caused by: java.io.IOException: The requested operation cannot be performed
    on a
     file with a user-mapped section open
            at java.io.RandomAccessFile.setLength(Native Method)
            at kafka.log.OffsetIndex.liftedTree2$1(OffsetIndex.scala:263)
            at kafka.log.OffsetIndex.resize(OffsetIndex.scala:262)
            at kafka.log.OffsetIndex.trimToValidSize(OffsetIndex.scala:247)
            at kafka.log.Log.rollToOffset(Log.scala:518)
            at kafka.log.Log.roll(Log.scala:502)
            at kafka.log.Log.maybeRoll(Log.scala:484)
            at kafka.log.Log.append(Log.scala:297)

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/lizziew/kafka 0.8

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/6.patch

----

----

Re: kafka pull request: Unmap before resizing

[Jay Kreps <ja...@gmail.com>]

Hi, can you file a JIRA and attach the patch there? This does the Apache
copyright stuff...

Instructions here:
http://kafka.apache.org/contributing.html

This seems like a good thing to have. Is there a more portable way to do
this? Obviously this would not work on a non-sun JVM. I think the ideal
would be to add a tryUnmap() method that attempts the case but swallows any
exception.

Also I would like to fully understand the Sun's objection to allowing user
code to unmap buffers. In that ticket on the previous thread on
Windows+mmap there was a mention that forcing the unmap was somehow a
security hole. I don't understand this. Unix obviously supports unmap and
there is no security issue with that as normal process-level memory
isolation protects processes from reading each others memory or files.  I
suspect maybe this would only be true in cases where you are trying to
isolate different code in the same JVM which is not a valid use case. But I
want to fully understand the issue before we add something someone claims
is unsafe.

-Jay


On Thu, Jul 25, 2013 at 12:07 AM, lizziew <gi...@git.apache.org> wrote:

> GitHub user lizziew opened a pull request:
>
>     https://github.com/apache/kafka/pull/6
>
>     Unmap before resizing
>
>     While I was studying how MappedByteBuffer works, I saw a sharing
> runtime exception on Windows. I applied what I learned to generate a patch
> which uses an internal open JDK API to solve this problem.
>
>     ---
>
>     Caused by: java.io.IOException: The requested operation cannot be
> performed
>     on a
>      file with a user-mapped section open
>             at java.io.RandomAccessFile.setLength(Native Method)
>             at kafka.log.OffsetIndex.liftedTree2$1(OffsetIndex.scala:263)
>             at kafka.log.OffsetIndex.resize(OffsetIndex.scala:262)
>             at kafka.log.OffsetIndex.trimToValidSize(OffsetIndex.scala:247)
>             at kafka.log.Log.rollToOffset(Log.scala:518)
>             at kafka.log.Log.roll(Log.scala:502)
>             at kafka.log.Log.maybeRoll(Log.scala:484)
>             at kafka.log.Log.append(Log.scala:297)
>
> You can merge this pull request into a Git repository by running:
>
>     $ git pull https://github.com/lizziew/kafka 0.8
>
> Alternatively you can review and apply these changes as the patch at:
>
>     https://github.com/apache/kafka/pull/6.patch
>
> ----
>
> ----
>
>