You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by ro...@apache.org on 2020/03/31 22:50:04 UTC
[beam] 01/01: Merge pull request #11203 [BEAM-9577] Define and
implement dependency-aware artifact staging service.
This is an automated email from the ASF dual-hosted git repository.
robertwb pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
commit dbc21d97de83de08568ddd45ba5ec21d3f83d118
Merge: 2f12213 b2ad0f9
Author: Robert Bradshaw <ro...@google.com>
AuthorDate: Tue Mar 31 15:49:48 2020 -0700
Merge pull request #11203 [BEAM-9577] Define and implement dependency-aware artifact staging service.
.../src/main/proto/beam_provision_api.proto | 2 +-
.../src/main/proto/beam_artifact_api.proto | 92 +-
.../core/construction/ArtifactServiceStager.java | 16 +-
.../InMemoryArtifactStagerService.java | 8 +-
.../runners/flink/FlinkJobServerDriverTest.java | 2 +-
...=> AbstractLegacyArtifactRetrievalService.java} | 22 +-
...a => AbstractLegacyArtifactStagingService.java} | 13 +-
...mFileSystemLegacyArtifactRetrievalService.java} | 17 +-
...eamFileSystemLegacyArtifactStagingService.java} | 22 +-
...ClassLoaderLegacyArtifactRetrievalService.java} | 11 +-
...avaFilesystemLegacyArtifactStagingService.java} | 11 +-
...ce.java => LegacyArtifactRetrievalService.java} | 2 +-
.../control/DefaultJobBundleFactory.java | 18 +-
.../environment/DockerEnvironmentFactory.java | 10 +-
.../environment/EmbeddedEnvironmentFactory.java | 4 +-
.../environment/EnvironmentFactory.java | 4 +-
.../environment/ExternalEnvironmentFactory.java | 10 +-
.../environment/ProcessEnvironmentFactory.java | 10 +-
.../StaticRemoteEnvironmentFactory.java | 4 +-
.../fnexecution/jobsubmission/JobServerDriver.java | 19 +-
.../jobsubmission/PortablePipelineJarCreator.java | 29 +-
.../BeamFileSystemArtifactServicesTest.java | 72 +-
.../artifact/ClassLoaderArtifactServiceTest.java | 84 +-
.../control/DefaultJobBundleFactoryTest.java | 6 +-
.../environment/DockerEnvironmentFactoryTest.java | 4 +-
.../environment/ProcessEnvironmentFactoryTest.java | 4 +-
.../beam/runners/samza/SamzaJobServerDriver.java | 4 +-
sdks/go/cmd/beamctl/cmd/artifact.go | 4 +-
sdks/go/pkg/beam/artifact/gcsproxy/retrieval.go | 2 +-
sdks/go/pkg/beam/artifact/gcsproxy/staging.go | 4 +-
sdks/go/pkg/beam/artifact/materialize.go | 12 +-
sdks/go/pkg/beam/artifact/materialize_test.go | 8 +-
sdks/go/pkg/beam/artifact/server_test.go | 8 +-
sdks/go/pkg/beam/artifact/stage.go | 10 +-
sdks/go/pkg/beam/artifact/stage_test.go | 8 +-
.../model/fnexecution_v1/beam_provision_api.pb.go | 2 +-
.../model/jobmanagement_v1/beam_artifact_api.pb.go | 929 +++++++++++++++++----
.../pkg/beam/runners/universal/runnerlib/stage.go | 2 +-
sdks/python/.pylintrc | 1 +
.../runners/portability/abstract_job_service.py | 2 +-
.../runners/portability/artifact_service.py | 263 +++++-
.../runners/portability/artifact_service_test.py | 170 +++-
.../runners/portability/fn_api_runner/fn_runner.py | 4 -
.../portability/fn_api_runner/worker_handlers.py | 8 +-
.../runners/portability/local_job_service.py | 2 +-
.../runners/portability/local_job_service_test.py | 2 +-
.../runners/portability/portable_stager.py | 14 +-
.../runners/portability/portable_stager_test.py | 11 +-
.../portability/spark_uber_jar_job_server_test.py | 2 +-
49 files changed, 1552 insertions(+), 416 deletions(-)