[jira] [Commented] (BEAM-455) Implement a test harness for verifying correctness of custom sources

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15379826#comment-15379826 ] 

ASF GitHub Bot commented on BEAM-455:
-------------------------------------

GitHub user chamikaramj opened a pull request:

    https://github.com/apache/incubator-beam/pull/667

    [BEAM-455] Adds a test harnesses and utilities framework for sources.

    Helper functions and test harnesses for checking correctness of source (``iobase.BoundedSource``) and range tracker (``iobase.RangeTracker``) implementations.
    
    Contains a few lightweight utilities (e.g. reading items from a source such as ``readFromSource()``, as well as heavyweight property testing and stress testing harnesses that help getting a large amount of test
    coverage with few code.
    
    Most notable ones are:
    * ``assertSourcesEqualReferenceSource()`` helps testing that the data read by the union of sources   produced by ``BoundedSource.split()`` is the same as data read by the original source.
    * If your source implements dynamic work rebalancing, use the ``assertSplitAtFraction()`` family of functions - they test behavior of ``RangeTracker.try_split()``, in particular, that various consistency properties are respected and the total set of data read by the source is preserved when splits happen. 
    Use ``assertSplitAtFractionBehavior()`` to test individual cases of dynamic work rebalancing and use ``assertSplitAtFractionExhaustive()`` as a heavy-weight stress test including concurrency.
    
    Updates 'avroio_test.py' to use the test framework.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/chamikaramj/incubator-beam testingframework_l

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/667.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #667
    
----
commit b3ac40ecca86f5952bd5ac3799e55e3e2a1a72a6
Author: Chamikara Jayalath <ch...@google.com>
Date:   2016-07-15T18:07:48Z

    Adds a test harnesses and utilities framework for sources.
    
    Helper functions and test harnesses for checking correctness of source
    (``iobase.BoundedSource``) and range tracker (``iobase.RangeTracker``)
    implementations.
    
    Contains a few lightweight utilities (e.g. reading items from a source
    such as ``readFromSource()``, as well as heavyweight property testing
    and stress testing harnesses that help getting a large amount of test
    coverage with few code.
    
    Most notable ones are:
    * ``assertSourcesEqualReferenceSource()`` helps testing that the data
    read by the union of sources produced by ``BoundedSource.split()`` is
    the same as data read by the original source.
    * If your source implements dynamic work rebalancing, use the
    ``assertSplitAtFraction()`` family of functions - they test behavior of
    ``RangeTracker.try_split()``, in particular, that various consistency
    properties are respected and the total set of data read by the source
    is preserved when splits happen. Use ``assertSplitAtFractionBehavior()``
    to test individual cases of ``splitAtFraction()`` and use
    ``assertSplitAtFractionExhaustive()`` as a heavy-weight stress test
    including concurrency.

----


> Implement a test harness for verifying correctness of custom sources
> --------------------------------------------------------------------
>
>                 Key: BEAM-455
>                 URL: https://issues.apache.org/jira/browse/BEAM-455
>             Project: Beam
>          Issue Type: New Feature
>          Components: sdk-py
>            Reporter: Chamikara Jayalath
>            Assignee: Chamikara Jayalath
>
> Verifying correctness of custom sources can be tricky, for example, due to  complex interactions patterns that can occur when performing dynamic work rebalancing.
> Having a test harness with utilities for easily testing sources will be extremely useful for users.
> The proposed test harness will be similar to the one that is already available for Java SDK [1] but will be adjusted to match the Python custom source API.
> [1] https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/SourceTestUtils.java



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)