[jira] [Work logged] (BEAM-13709) PipelineOptions() and from_dictionary parsing use_public_ips and no_use_public_ips differently

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-13709?focusedWorklogId=751795&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-751795 ]

ASF GitHub Bot logged work on BEAM-13709:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 01/Apr/22 20:54
            Start Date: 01/Apr/22 20:54
    Worklog Time Spent: 10m 
      Work Description: AnandInguva closed pull request #16715:
URL: https://github.com/apache/beam/pull/16715


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 751795)
    Time Spent: 8h 40m  (was: 8.5h)

> PipelineOptions() and from_dictionary parsing use_public_ips and no_use_public_ips differently
> ----------------------------------------------------------------------------------------------
>
>                 Key: BEAM-13709
>                 URL: https://issues.apache.org/jira/browse/BEAM-13709
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-py-core
>            Reporter: Minbo Bae
>            Assignee: Anand Inguva
>            Priority: P2
>              Labels: starter, usability
>             Fix For: 2.38.0
>
>          Time Spent: 8h 40m
>  Remaining Estimate: 0h
>
> {{PipelineOptions}} in Python has two methods to pass a param dict: using in constructor {{PipelineOptions(**params)}} or {{{}PipelineOptions.from_dictionary(params){}}}.
> But, they work slightly differently:
>  * [PipelineOptions(**params)|https://github.com/apache/beam/blob/v2.35.0/sdks/python/apache_beam/options/pipeline_options.py#L313-L324] discards an option if it is not defined as a dest of {{argparse}} in an Option class. For example, {{no_use_public_ips=True}} is ignored and the Dataflow job will run with public IPs. To disable public IPs, the option dictionary must use {{{}use_public_ips{}}}.
>  * [PipelineOptions.from_dictionary()|https://github.com/apache/beam/blob/v2.35.0/sdks/python/apache_beam/options/pipeline_options.py#L229] skips an option if the option value is {{{}False{}}}. For example, {{use_public_ips=False}} is ignored and the Dataflow job will run with public IPs. To disable public IPs, the option dictionary must use {{no_use_public_ips.}}
> This makes the user very confused, and sometimes the pipeline works in an unexpected way. 
> We must have the consistent behavior between the two methods, or at least a warning about invalid ignored options.
> BEAM-9093 dealt with a similar issue for {{PipelineOptions()}}. Like the issue, I guess adding a warning in `PipelineOptions.from_dictionary()` for ignored options can help reducing the confusion, if we cannot have two methods have exactly the same behavior.
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)