[jira] [Updated] (AIRFLOW-6350) security - spark submit operator logging+exceptions should mask passwords

["Jarek Potiuk (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jarek Potiuk updated AIRFLOW-6350:
----------------------------------
    Fix Version/s:     (was: 2.0.0)
                   1.10.8

> security - spark submit operator logging+exceptions should mask passwords 
> --------------------------------------------------------------------------
>
>                 Key: AIRFLOW-6350
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-6350
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: hooks, operators
>    Affects Versions: 1.10.3
>            Reporter: t oo
>            Assignee: t oo
>            Priority: Major
>             Fix For: 1.10.8
>
>
> contrib/hooks/spark_submit_hook.py    
> Mask passwords in spark submit cmd AND error stacktrace
>  
> *add*
> def _mask_cmd(self, connection_cmd):
>  # Mask any password related fields in application args with key value pair where key contains password (case insensitive), e.g. HivePassword='abc'
> connection_cmd_masked = re.sub(r"(\S*?(?:secret|password)\S*?\s*=\s*')[^']*(?=')", r'\1******', ' '.join(connection_cmd), flags=re.I)
> return connection_cmd_masked
>  
> *BEFORE*
> self.log.info("Spark-Submit cmd: %s", connection_cmd)
>  
> *AFTER*
> self.log.info("Spark-Submit cmd: %s", self._mask_cmd(connection_cmd))
>  
> *BEFORE*
>  if returncode or (self._is_kubernetes and self._spark_exit_code != 0):
> raise AirflowException(
> "Cannot execute: {}. Error code is: {}.".format(
> spark_submit_cmd, returncode
> )
> )
>  
> *AFTER*
> if returncode or (self._is_kubernetes and self._spark_exit_code != 0):
> raise AirflowException(
> "Cannot execute: {}. Error code is: {}.".format(
> self._mask_cmd(spark_submit_cmd), returncode
> )
> )



--
This message was sent by Atlassian Jira
(v8.3.4#803005)