You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by ib...@apache.org on 2020/01/03 21:36:54 UTC

[beam] branch master updated (0851849 -> 9bceff5)

This is an automated email from the ASF dual-hosted git repository.

ibzib pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


    from 0851849  [BEAM-5605] Ensure that split calls are routed to the active bundle processor for the bundle id. (#10489)
     new bf6577e  Import freezegun for Python time testing.
     new b329de6  Allow message stream to yield duplicates.
     new 4479865  [BEAM-8891] Create and submit Spark portable jar in Python.
     new 6f7b2b2  Refactor shared uber jar generation code into common subclass.
     new 4996c14  Make Spark REST URL a separate pipeline option.
     new 9bceff5  Merge pull request #10403 from ibzib/spark-jar

The 24863 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../beam/runners/spark/SparkPipelineRunner.java    |   4 +
 .../python/apache_beam/options/pipeline_options.py |  22 ++
 .../runners/portability/abstract_job_service.py    |  78 ++++++-
 .../portability/flink_uber_jar_job_server.py       |  71 +-----
 .../runners/portability/portable_runner.py         |  11 +-
 .../runners/portability/spark_runner.py            |  30 ++-
 .../portability/spark_uber_jar_job_server.py       | 255 +++++++++++++++++++++
 .../portability/spark_uber_jar_job_server_test.py  | 209 +++++++++++++++++
 sdks/python/setup.py                               |   1 +
 9 files changed, 593 insertions(+), 88 deletions(-)
 create mode 100644 sdks/python/apache_beam/runners/portability/spark_uber_jar_job_server.py
 create mode 100644 sdks/python/apache_beam/runners/portability/spark_uber_jar_job_server_test.py