[jira] [Commented] (AIRFLOW-3997) Add accessor for optional variables

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16782346#comment-16782346 ] 

ASF GitHub Bot commented on AIRFLOW-3997:
-----------------------------------------

kristiany commented on pull request #4819: [AIRFLOW-3997] Add accessor for optional variables
URL: https://github.com/apache/airflow/pull/4819
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   - [x] My PR addresses the following [Airflow Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
     - https://issues.apache.org/jira/browse/AIRFLOW-3997
     - In case you are fixing a typo in the documentation you can prepend your commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
     - In case you are proposing a fundamental code change, you need to create an Airflow Improvement Proposal([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
   
   ### Description
   
   - [x] Here are some details about my PR, including screenshots of any UI changes:
   
   Add a `optional` function for a safe way to retrieve a variable, without exploding if it doesn't exist.  
   
   This will not change existing functions in the `Variable` class. If
   variable `foo` doesn't exist:
   ```
   foo = Variable.get("foo")
   -> KeyError
   ```
   
   For optional `None` is returned instead:
   ```
   foo = Variable.optional("foo")
   if foo is None:
       handle_missing_foo()
   ```
   
   * Add optional accessor to the `Variable` class with support for JSON deserialization and session
   
   
   ### Tests
   
   - [x] My PR adds the following unit tests __OR__ does not need testing for this extremely good reason:
   
   * Test the optional accessor with support for JSON deserialization
   * Test the current behaviour for missing variables for the `get` function to raise a `KeyError`
   
   ### Commits
   
   - [x] My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
     1. Subject is separated from body by a blank line
     1. Subject is limited to 50 characters (not including Jira issue reference)
     1. Subject does not end with a period
     1. Subject uses the imperative mood ("add", not "adding")
     1. Body wraps at 72 characters
     1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [x] In case of new functionality, my PR adds documentation that describes how to use it.
     - When adding new operators/hooks/sensors, the autoclass documentation generation needs to be added.
     - All the public functions and the classes in the PR contain docstrings that explain what it does
   
   Describe the optional accessor in the doc, in the get variable examples section. Also describe the current behaviour for `get` of raising an error when a variable is missing
   
   ### Code Quality
   
   - [ ] Passes `flake8`
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Add accessor for optional variables
> -----------------------------------
>
>                 Key: AIRFLOW-3997
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3997
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: models
>    Affects Versions: 1.10.2
>            Reporter: Kristian Yrjölä
>            Assignee: Kristian Yrjölä
>            Priority: Minor
>
> h4. TL;DR
> Proposing an addition to the _Variable_ accessors for optional variables. No change in existing functions.
>  
> h4. Description
> Add a function to get a variable if it exists, and don't raise an error if it doesn't.
> *Rationale:*
> * The get function raises a _KeyError_ for a missing variable, if not careful that can bring down the whole application. Having a optional get function can make client code safer, isolating errors to the code using the specific variable only
> * Sometimes there is no obvious value to set as default, but you rather want to know if the variable is missing
> * The current get function doesn't support _None_ as a default value
> * Optional variables are good for feature flags for instance if you want to control certain behaviours between environments before it's released in production in a continuous deployment setup
> Passing in _default_var=None_ still raises a _KeyError_ since that is the default value of the _default_var_ parameter. If variable "foo" doesn't exist:
> {code:java}
> foo = Variable.get("foo", default_var=None)
> -> KeyError{code}
> You could use empty string or other values to indicate a missing variable, like this:
> {code:java}
> foo = Variable.get("foo", default_var="n/a")
> if foo == "n/a":
>  handle_missing_foo(){code}
> *My proposal is this:*
> {code:java}
> foo = Variable.optional("foo")
> if foo is None:
>  handle_missing_foo(){code}
> Which IMHO is more in line with Python's handling of a missing value.
> The function should still support the _serialize_json_ and _session_ parameters, but not the _default_var_ parameter, since that would make little sense.
> Of course this can still be handled on the client side with a try-catch. However it can be a natural part of the API for the _Variable_ class, replacing many try-catches in client code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)