[jira] [Created] (AIRFLOW-5861) DatabricksRunNowOperator enforcing argument that should be able to be empty

["John Dyer (Jira)" <ji...@apache.org>]

John Dyer created AIRFLOW-5861:
----------------------------------

             Summary: DatabricksRunNowOperator enforcing argument that should be able to be empty
                 Key: AIRFLOW-5861
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5861
             Project: Apache Airflow
          Issue Type: Bug
          Components: operators
    Affects Versions: 1.10.6
            Reporter: John Dyer


It looks like /airflow/utils/decorators.py: apply_defaults makes sure that arguments without a default value are enforced.  In the DatabricksRunNowOperator, the documentation specifies:
{noformat}
There are two ways to instantiate this operator.

    In the first way, you can take the JSON payload that you typically use
    to call the ``api/2.0/jobs/run-now`` endpoint and pass it directly
    to our ``DatabricksRunNowOperator`` through the ``json`` parameter.
    For example ::

        json = {
          "job_id": 42,
          "notebook_params": {
            "dry-run": "true",
            "oldest-time-to-consider": "1457570074236"
          }
        }

        notebook_run = DatabricksRunNowOperator(task_id='notebook_run', json=json){noformat}
But, when we run it that way, we get:
{noformat}
Broken DAG: [/home/airflow-user/my-awesome-dag.py] Argument ['job_id'] is required{noformat}
So, my thought to fix this is to provide a default for job_id as None, here:
[https://github.com/apache/airflow/blob/v1-10-stable/airflow/contrib/operators/databricks_operator.py#L433]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)