[GitHub] incubator-beam pull request #1694: [BEAM-625] A few memory optimizations in ...

[katsiapis <gi...@git.apache.org>]

GitHub user katsiapis opened a pull request:

    https://github.com/apache/incubator-beam/pull/1694

    [BEAM-625] A few memory optimizations in Avro.

    - Using a buffer when decompressing with Snappy in Avro in order to avoid  \u2026
    unnecessary copies during slicing of large strings.
    - Explicitly gc-ing (possibly large) Avro Block with its associated data.
    - Removing somewhat confusing cStringIO aliasing.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/katsiapis/incubator-beam memory

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/1694.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1694
    
----
commit 069ceebcb167a6c3976b362548a5d91c003a3894
Author: Gus Katsiapis <ka...@katsiapis-linux.mtv.corp.google.com>
Date:   2016-12-26T20:13:06Z

    Using a buffer when decompressing with Snappy in Avro in order to avoid
    unnecessary copies during slicing of large strings.

commit 4f5e9b51740a97df692138fa477a9f6c66833ab1
Author: Gus Katsiapis <ka...@katsiapis-linux.mtv.corp.google.com>
Date:   2016-12-26T20:17:57Z

    Explicitly gc-ing (possibly large) Avro Block with its associated data.

commit e91662bef5773bc23ca2619373496c632693e4d5
Author: Gus Katsiapis <ka...@katsiapis-linux.mtv.corp.google.com>
Date:   2016-12-26T20:26:03Z

    Removing somewhat confusing cStringIO aliasing.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---