[jira] [Resolved] (AIRFLOW-7024) Add the verbose parameter support to SparkSqlOperator

["Kaxil Naik (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-7024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kaxil Naik resolved AIRFLOW-7024.
---------------------------------
    Fix Version/s: 1.10.10
       Resolution: Fixed

> Add the verbose parameter support to SparkSqlOperator
> -----------------------------------------------------
>
>                 Key: AIRFLOW-7024
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-7024
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: operators
>    Affects Versions: 1.10.9
>            Reporter: Kengo Seki
>            Assignee: Kengo Seki
>            Priority: Major
>             Fix For: 1.10.10
>
>
> The docstring for SparkSqlOperator says that it supports a {{verbose}} parameter, but actually it doesn't.
> {code:title=airflow/providers/apache/spark/operators/spark_sql.py}
> class SparkSqlOperator(BaseOperator):
>     """
>     (snip)
>     :param verbose: Whether to pass the verbose flag to spark-sql
>     :type verbose: bool
>     :param yarn_queue: The YARN queue to submit to (Default: "default")
>     :type yarn_queue: str
>     """
>     (snip)
>     @apply_defaults
>     def __init__(self,
>                  sql,
>                  conf=None,
>                  conn_id='spark_sql_default',
>                  total_executor_cores=None,
>                  executor_cores=None,
>                  executor_memory=None,
>                  keytab=None,
>                  principal=None,
>                  master='yarn',
>                  name='default-name',
>                  num_executors=None,
>                  yarn_queue='default',
>                  *args,
>                  **kwargs):
>         super().__init__(*args, **kwargs)
>         self._sql = sql
>         self._conf = conf
>         self._conn_id = conn_id
>         self._total_executor_cores = total_executor_cores
>         self._executor_cores = executor_cores
>         self._executor_memory = executor_memory
>         self._keytab = keytab
>         self._principal = principal
>         self._master = master
>         self._name = name
>         self._num_executors = num_executors
>         self._yarn_queue = yarn_queue
>         self._hook = None
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)