[jira] [Created] (BEAM-10671) Add environment configuration fields as first-class pipeline options

["Kyle Weaver (Jira)" <ji...@apache.org>]

Kyle Weaver created BEAM-10671:
----------------------------------

             Summary: Add environment configuration fields as first-class pipeline options
                 Key: BEAM-10671
                 URL: https://issues.apache.org/jira/browse/BEAM-10671
             Project: Beam
          Issue Type: Improvement
          Components: sdk-py-harness
            Reporter: Kyle Weaver
            Assignee: Kyle Weaver


The pipeline option --environment_config has completely different usages depending on the value of --environment_type. This is confusing for the user. Additionally, --environment_config is a JSON blob for --environment_type=PROCESS. This JSON blob is a huge pain to escape and pass around compared to a collection of flat strings.

We should replace --environment_config with first-class / top-level pipeline options for each environment type:

DOCKER

--environment_container_image

PROCESS

--environment_os

--environment_architecture

--environment_variables

EXTERNAL

--environment_service_address

LOOPBACK

(none)

This way we can validate that the user is configuring these options correctly (ie give a warning or error if they use options that do not apply to their chosen --environment_type).

We can deprecate the --environment_config option, logging a warning until removing this option altogether in a future Beam release.

[https://beam.apache.org/documentation/runtime/sdk-harness-config/]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)