You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by dhalperi <gi...@git.apache.org> on 2016/08/05 16:07:36 UTC

[GitHub] incubator-beam pull request #794: Compressed source test and cleanups

GitHub user dhalperi opened a pull request:

    https://github.com/apache/incubator-beam/pull/794

    Compressed source test and cleanups

    R: @jkff @bjchambers 

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dhalperi/incubator-beam compressed-source-test

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/794.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #794
    
----
commit 960e8db24aa96be812d81dc86c75ededddcc9a0d
Author: Dan Halperin <dh...@google.com>
Date:   2016-08-05T15:51:15Z

    TextIO: test compression with larger files
    
    It is important for correctness that we test with large files
    because otherwise the compressed file may be larger than the
    uncompressed file, which could mask bugs

commit 8a83165fff9e419fd828499243b3055079791bed
Author: Dan Halperin <dh...@google.com>
Date:   2016-08-05T16:05:55Z

    CompressedSource: CompressedReader is never splittable
    
    The only way it's safe to split a compressed file is if the file is not compressed. This can
    only happen when the source itself is splittable, and that in turn will result in the inner
    source's reader being returned. A CompressedReader will only be created in the event that
    the file is NOT splittable. So remove all the logic handling splittable compressed readers,
    and instead go with the logic when we know/assume the file is compressed.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-beam pull request #794: Compressed source test and cleanups

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/incubator-beam/pull/794


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---