[jira] [Commented] (FLINK-8941) SpanningRecordSerializationTest fails on Travis

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-8941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16401936#comment-16401936 ] 

ASF GitHub Bot commented on FLINK-8941:
---------------------------------------

GitHub user NicoK opened a pull request:

    https://github.com/apache/flink/pull/5709

    [FLINK-8941][network][serializer] improve SpanningRecordSerializationTest and ensure unique spilling files

    ## What is the purpose of the change
    
    This PR contains two commits trying to tackle FLINK-8941 (which I could not reproduce).
    
    ## Brief change log
    
    - let `SpanningRecordSerializationTest` extend from `TestLogger`
    - use a `TemporaryFolder` in `SpanningRecordSerializationTest` for spilling files
    - make sure `SpillingAdaptiveSpanningRecordDeserializer` does not work on an existing file, e.g. from another instance running on the same machine - allow 10 retries with 20 random bytes file names and fail otherwise
    
    ## Verifying this change
    
    This change is already covered by existing tests, such as `SpanningRecordSerializationTest`.
    
    ## Does this pull request potentially affect one of the following parts:
    
      - Dependencies (does it add or upgrade a dependency): **no**
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **no**
      - The serializers: **yes** (well, partly - the code around the actual serializers)
      - The runtime per-record code paths (performance sensitive): **no**
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: **no**
      - The S3 file system connector: **no**
    
    ## Documentation
    
      - Does this pull request introduce a new feature? **no**
      - If yes, how is the feature documented? **not applicable**


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/NicoK/flink flink-8941

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/5709.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5709
    
----
commit 5086956c743273dc4cd379fc6a6ee97da5c20258
Author: Nico Kruber <ni...@...>
Date:   2018-03-16T10:43:43Z

    [FLINK-8941][tests] use TestLogger and TemporaryFolder in SpanningRecordSerializationTest

commit c9ba08b1c92c2510676c3cd3116410c7a379d072
Author: Nico Kruber <ni...@...>
Date:   2018-03-16T10:51:29Z

    [FLINK-8941][serialization] make sure we use unique spilling files
    
    Although the spilling files were chosen with random names of 20 bytes, it could
    rarely happen that these collide. In that case, have another try (at most 10) at
    selecting a unique file name.

----


> SpanningRecordSerializationTest fails on Travis
> -----------------------------------------------
>
>                 Key: FLINK-8941
>                 URL: https://issues.apache.org/jira/browse/FLINK-8941
>             Project: Flink
>          Issue Type: Improvement
>          Components: Tests
>            Reporter: Chesnay Schepler
>            Assignee: Nico Kruber
>            Priority: Blocker
>              Labels: test-stability
>             Fix For: 1.5.0
>
>
> https://travis-ci.org/zentol/flink/jobs/353217791
> {code:java}
> testHandleMixedLargeRecords(org.apache.flink.runtime.io.network.api.serialization.SpanningRecordSerializationTest)  Time elapsed: 1.992 sec  <<< ERROR!
> java.nio.channels.ClosedChannelException: null
> 	at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:110)
> 	at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:199)
> 	at org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer$SpanningWrapper.addNextChunkFromMemorySegment(SpillingAdaptiveSpanningRecordDeserializer.java:528)
> 	at org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer$SpanningWrapper.access$200(SpillingAdaptiveSpanningRecordDeserializer.java:430)
> 	at org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer.setNextBuffer(SpillingAdaptiveSpanningRecordDeserializer.java:75)
> 	at org.apache.flink.runtime.io.network.api.serialization.SpanningRecordSerializationTest.testSerializationRoundTrip(SpanningRecordSerializationTest.java:143)
> 	at org.apache.flink.runtime.io.network.api.serialization.SpanningRecordSerializationTest.testSerializationRoundTrip(SpanningRecordSerializationTest.java:109)
> 	at org.apache.flink.runtime.io.network.api.serialization.SpanningRecordSerializationTest.testHandleMixedLargeRecords(SpanningRecordSerializationTest.java:98){code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)