[jira] [Created] (AIRFLOW-5819) AWSBatchOperator has invalid default value for array_properties

["Dom (Jira)" <ji...@apache.org>]

Dom created AIRFLOW-5819:
----------------------------

             Summary: AWSBatchOperator has invalid default value for array_properties
                 Key: AIRFLOW-5819
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5819
             Project: Apache Airflow
          Issue Type: Bug
          Components: operators
    Affects Versions: 1.10.6
            Reporter: Dom


We upgraded our Airflow version from 1.10.3 to 1.10.6 and saw our AWSBatchOperator throwing the following error (easily fixed by changing the default param from None to {}):
{code:java}
 [2019-10-30 13:10:15.392] INFO:airflow.task.operators.execute:93 Running AWS Batch Job - Job definition: batch-jobdef-1 - on queue AwsBatch-batch-queue
[2019-10-30 13:10:15.393] INFO:airflow.task.operators.execute:95 AWSBatchOperator overrides: {'command': ['--start_datetime', '2019-10-29']}
[2019-10-30 13:10:15.433] INFO:airflow.task.operators.execute:121 AWS Batch Job has failed executed
[2019-10-30 13:10:15.445] ERROR:airflow.task.handle_failure:1058 Parameter validation failed:
Invalid type for parameter arrayProperties, value: None, type: <class 'NoneType'>, valid types: <class 'dict'>
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/airflow/contrib/operators/awsbatch_operator.py", line 108, in execute
    containerOverrides=self.overrides)
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 634, in _make_api_call
    api_params, operation_model, context=request_context)
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 682, in _convert_to_request_dict
    api_params, operation_model)
  File "/usr/local/lib/python3.7/site-packages/botocore/validate.py", line 297, in serialize_to_request
    raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid type for parameter arrayProperties, value: None, type: <class 'NoneType'>, valid types: <class 'dict'>During handling of the above exception, another exception occurred:Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 930, in _run_raw_task
    result = task_copy.execute(context=context)
  File "/usr/local/lib/python3.7/site-packages/airflow/contrib/operators/awsbatch_operator.py", line 122, in execute
    raise AirflowException(e)
airflow.exceptions.AirflowException: Parameter validation failed:
Invalid type for parameter arrayProperties, value: None, type: <class 'NoneType'>, valid types: <class 'dict'>
[2019-10-30 13:10:15.447] INFO:airflow.task.handle_failure:1087 All retries failed; marking task as FAILED{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)