[jira] [Commented] (FLINK-3166) The first program in ObjectReuseITCase has the wrong expected result, and it succeeds

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15060984#comment-15060984 ] 

ASF GitHub Bot commented on FLINK-3166:
---------------------------------------

GitHub user greghogan opened a pull request:

    https://github.com/apache/flink/pull/1464

    [FLINK-3166] [runtime] The first program in ObjectReuseITCase has the wrong expected result, and it succeeds

    - TestEnvironment now honors configuration of object reuse
    - Fixed reduce transformations to allow the user to modify and return either input
    
    This is neither clean nor complete but I wanted to get some feedback before continuing.
    
    ObjectReuseITCase was forcing object reuse, this was fixed by updating TestEnvironment.
    
    ReduceCombineDriver and ReduceDriver both requests the next record and call the user's reduce() and so can handle whichever of the two inputs objects the user might write to.
    
    In the second test case DataSourceTask (which requests the next record) calls ChainedAllReduceDriver.collect (which calls the user's reduce()). There is no feedback so DataSourceTask must cycle through `n`+1 reusable objects for unknown `n`.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/greghogan/flink 3166_fix_objectreuseitcase

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/1464.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1464
    
----
commit 78dc073440ed8ecfec51fbe33ce665562786bf20
Author: Greg Hogan <co...@greghogan.com>
Date:   2015-12-16T19:42:06Z

    [FLINK-3166] [runtime] The first program in ObjectReuseITCase has the wrong expected result, and it succeeds
    - TestEnvironment now honors configuration of object reuse
    - Fixed reduce transformations to allow the user to modify and return either input

----


> The first program in ObjectReuseITCase has the wrong expected result, and it succeeds
> -------------------------------------------------------------------------------------
>
>                 Key: FLINK-3166
>                 URL: https://issues.apache.org/jira/browse/FLINK-3166
>             Project: Flink
>          Issue Type: Bug
>          Components: Distributed Runtime, Documentation, Tests
>            Reporter: Gabor Gevay
>            Assignee: Greg Hogan
>            Priority: Critical
>
> The first program in ObjectReuseITCase has the following input:
> a,1
> a,2
> a,3
> a,4
> a,50
> There is a groupBy on field 0, and then a reduce, so the result should be 1+2+3+4+50 = 60. But the hardcoded expected result is 100, and running the Flink program also produces this.
> The problem is caused my mismatched assumptions between ReduceCombineDriver.sortAndCombine and the ReduceFunction in the test about object reuse rules of ReduceFunctions:
> ReduceCombineDriver.sortAndCombine has the following comment:
> "The user function is expected to return the first input as the result."
> While the ReduceFunction in the test is modifying and returning the second input. (And the second program in the test also has the same problem.)
> I can't find the assumption that is stated in the comment in any documentation. For example, the javadoc of ReduceFunction should make the user aware of this. Or, alternatively, the code of the driver should be modified to not make this assumption. I'm not sure which solution is better.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)