You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by dhalperi <gi...@git.apache.org> on 2017/04/18 02:36:09 UTC

[GitHub] beam pull request #2563: [BEAM-59] FileBasedSource: convert to FileSystem fr...

GitHub user dhalperi opened a pull request:

    https://github.com/apache/beam/pull/2563

    [BEAM-59] FileBasedSource: convert to FileSystem from IOChannelFactory

    Accompanying changes:
    
    * And change all the FileBasedSource implementations in the SDK.
    * Drop a few FileBasedSource.readFromX functions that are not
      style-compliant.
    * Add a few allowsDynamicSplitting overrides to FileBasedReaders where
      it's needed.
    * Since the FileSystem API can preserve the size information from
      listing in external file systems, get rid of all the sampling logic
      in FileBasedSource -- we can always use the exact size.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dhalperi/beam convert-filebasedsource

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/beam/pull/2563.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2563
    
----
commit ecf7082fc034bbeb5f6980ad581ce0683934a7f6
Author: Dan Halperin <dh...@google.com>
Date:   2017-04-14T15:52:42Z

    FileBasedSource: convert to FileSystem from IOChannelFactory
    
    Other stuff:
    
    * And change all the FileBasedSource implementations in the SDK.
    * Drop a few FileBasedSource.readFromX functions that are not
      style-compliant.
    * Add a few allowsDynamicSplitting overrides to FileBasedReaders where
      it's needed.
    * Since the FileSystem API can preserve the size information from
      listing in external file systems, get rid of all the sampling logic
      in FileBasedSource -- we can always use the exact size.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #2563: [BEAM-59] FileBasedSource: convert to FileSystem fr...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/beam/pull/2563


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---