You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by tomwhite <gi...@git.apache.org> on 2016/03/10 12:30:12 UTC

[GitHub] incubator-beam pull request: Import Spark Runner code

GitHub user tomwhite opened a pull request:

    https://github.com/apache/incubator-beam/pull/37

    Import Spark Runner code

    This addresses https://issues.apache.org/jira/browse/BEAM-6.
    
    I've preserved git history (using @mxm's amazing git rewriting trick from #12). This is just an initial import - the Spark runner build is not yet integrated with the main build, packages need changing, etc. That's going to take more work, so it might be a good idea to get this merged first.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/tomwhite/incubator-beam beam-6-import-spark-runner

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/37.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #37
    
----
commit ad444470e97f7a1a9bc4582831f9d6df334e9949
Author: Josh Wills <jw...@cloudera.com>
Date:   2014-07-11T01:59:48Z

    Initial commit

commit 845a8178ab8bb620d0278b8312f04dce3f1348d8
Author: Josh Wills <jw...@cloudera.com>
Date:   2014-07-11T04:25:35Z

    Dumbest proof of concept possible

commit cb7c86642ea9839e3638c16849e113cb3536604a
Author: Josh Wills <jw...@cloudera.com>
Date:   2014-11-04T00:40:14Z

    First bit of work to get this running against the new Dataflow API

commit dce03e4b7d607bf8ac9846fa4bfb26a81d747a63
Author: Josh Wills <jw...@cloudera.com>
Date:   2014-12-02T01:58:30Z

    Update version of dataflow to get new API method access

commit 08e94b28ea06667049817def8aa5aee7aa31ebe1
Author: Josh Wills <jw...@cloudera.com>
Date:   2014-12-02T02:23:05Z

    Add support for getters and a Flatten impl

commit 9fdac6ccb750488cdff8e3a696d114911d385ee4
Author: Josh Wills <jw...@cloudera.com>
Date:   2014-12-02T06:54:28Z

    Such code. Much features.

commit deca2c0bdbe699099717591fc89afa4175c5438c
Author: Josh Wills <jw...@cloudera.com>
Date:   2014-12-02T23:28:15Z

    Adding some more operators: toiterable, seqdo

commit 6ee38b2a0677fc8eb77b4cd0aca64cfec2ec7fd9
Author: Josh Wills <jw...@cloudera.com>
Date:   2014-12-03T00:31:31Z

    Support for ParDo.BoundMulti

commit 64c6d8d723023aafc2ee5754d41e00eca1524ddf
Author: Josh Wills <jw...@cloudera.com>
Date:   2014-12-03T00:54:14Z

    Fix bug in deserializing side inputs

commit 40adbeceb32b5d8a2a01bb5156c4e97ae9cfc58c
Author: Josh Wills <jw...@cloudera.com>
Date:   2014-12-03T03:13:24Z

    Add SparkRuntimeContext for handling shared runtime objects

commit 565509dc50a0f216665e6900a65842509b63e4d5
Author: Josh Wills <jw...@cloudera.com>
Date:   2014-12-03T04:12:49Z

    First cut at aggregators

commit bb219d48a89f4f61d090b7cf47566dcee85adcf9
Author: Josh Wills <jw...@cloudera.com>
Date:   2014-12-03T05:03:35Z

    First minimally working aggregators

commit 9152769c79c00aa8c9332c23ffcdd42a9876f402
Author: Josh Wills <jw...@cloudera.com>
Date:   2014-12-09T06:40:34Z

    Updates for 141206 SDK release

commit 3bd04ae9ebcd89c93e5e635d806b613c5c5438ae
Author: Josh Wills <jw...@cloudera.com>
Date:   2014-12-09T20:06:28Z

    Dummy impls of windowing-related ProcContext functions

commit ba74f19ba6bda500a9541bbd84655336f1cff149
Author: Juliet Hougland <ju...@cloudera.com>
Date:   2014-12-09T20:16:13Z

    Simplify pom.xml

commit 6aa08e067d87ddb896df5a1a828bc0c526b9964f
Author: Josh Wills <jw...@cloudera.com>
Date:   2014-12-09T23:30:29Z

    Add proper coder handling to RDD retrieval

commit 45be5086820ba71dbee81bd7785f81e38f899fe8
Author: Juliet Hougland <ju...@cloudera.com>
Date:   2014-12-10T00:35:25Z

    Refactor aggregation related classes.

commit 137d54a1efe4e134cc7645c01eb013db10d2be30
Author: Juliet Hougland <ju...@cloudera.com>
Date:   2014-12-10T00:37:46Z

    Adds Javadoc and Tests to project.

commit 67cf364252fcf1df66ad97dac12d0cc006aa5d46
Author: Juliet Hougland <ju...@cloudera.com>
Date:   2014-12-11T20:12:45Z

    Add README.md and update project version in pom.xml.

commit b9545895738856693f3b5a40e127e348763a91f8
Author: Juliet Hougland <ju...@cloudera.com>
Date:   2014-12-11T22:28:00Z

    Add apache2 license and cloudera copyright.

commit 1523ffdae2a2fdf8029dfab68da1430edc6fe951
Author: Juliet Hougland <ju...@cloudera.com>
Date:   2014-12-12T00:40:28Z

    Adds custom checkstyle.
    
    more checkstyle improvements

commit 2992838fe35e06f280fbbf2eb419973be7699281
Author: Juliet Hougland <ju...@cloudera.com>
Date:   2014-12-12T00:56:01Z

    Factor out tranform translation logic in to its own class.

commit 2e3fe1ab9412be1b10659cd273a4de0ec5f103cd
Author: Sean Owen <sr...@gmail.com>
Date:   2014-12-13T16:03:39Z

    Specify and rationalize generic types in State, CoderHelpers to start

commit 74892633fa1c5a66f2f05cbce4b0e571ed72d71f
Author: Juliet Hougland <ju...@cloudera.com>
Date:   2014-12-16T21:15:05Z

    Add simple word count test.

commit f9e8fab69223582ab5b68cdfb634fad4c0c525ef
Author: Juliet Hougland <ju...@cloudera.com>
Date:   2014-12-17T01:34:57Z

    Factor out spark pipeline options.

commit ec172bac7b77774f86f0982d0261e1b7bb6d52e6
Author: Sean Owen <sr...@gmail.com>
Date:   2014-12-19T21:37:23Z

    Miscellaneous inspection changes from IntelliJ

commit ed1e2f7fb66f0746c1a8511b398f41a6df0e1513
Author: Sean Owen <sr...@gmail.com>
Date:   2014-12-20T08:36:13Z

    Issue #13 : attempt to remove all generics warnings, or handle them explicitly

commit 225f6c0121e6171b02097cac86724920e0e32b50
Author: Sean Owen <sr...@gmail.com>
Date:   2014-12-21T18:31:44Z

    Update and specify POM plugin config; Update Spark to 1.1.1, JUnit to 4.12, Spark 1.2; Add source, javadoc plugins and other info; Fix javadoc errors and a few typos

commit 1f9cd04591497231a65c83d6d3f4a27428be79ac
Author: Juliet Hougland <ju...@cloudera.com>
Date:   2014-12-24T02:55:18Z

    Improve readme to explain current state of the repo, and to encourage outside contributions.

commit ba4b32687c1c1bbf86bec6fc3a87e9c90c078ad7
Author: Juliet Hougland <ju...@cloudera.com>
Date:   2014-12-23T18:44:46Z

    Update version of dataflow we depend on.
    
    The primary change needed to accomodate the new dataflow api is to how we
    handle side inputs.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-beam pull request: Import Spark Runner code

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/incubator-beam/pull/37


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---