[jira] [Commented] (BEAM-980) Ability to configure the DAG created by Apex Runner

["Thomas Weise (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15837916#comment-15837916 ] 

Thomas Weise commented on BEAM-980:
-----------------------------------

Apex runner does not currently expose the ability for the user to tweak the settings that influence physical plan and execution layer (these are called "attributes" in Apex). Ultimately the pipeline author knows what individual operators do and should be able to specify/package a configuration file that the Apex runner will pick up and pass on to the Apex engine to apply to its physical plan.


> Ability to configure the DAG created by Apex Runner
> ---------------------------------------------------
>
>                 Key: BEAM-980
>                 URL: https://issues.apache.org/jira/browse/BEAM-980
>             Project: Beam
>          Issue Type: Task
>          Components: runner-apex
>            Reporter: Thomas Weise
>            Assignee: Thomas Weise
>
> The Beam pipeline is translated to an Apex DAG of operators that have names that are derived from the transforms. In case of composite transforms those look like path names. Apex lets the user configure things like memory, vcores, parallelism through properties/attributes that reference the operator names. The configuration approach needs to be documented and supplemented with an example.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)