[GitHub] [airflow] eladkal edited a comment on issue #18010: Airflow scheduler with statsd enabled crashes when dag_id contains unexpected characters

[GitBox <gi...@apache.org>]

eladkal edited a comment on issue #18010:
URL: https://github.com/apache/airflow/issues/18010#issuecomment-912808261


   >Agree it's likely something we should address. I am just wondering why it did not fail at trying to insert DAG to the DB - the DAG_ID there should only allow ASCII.
   
   I was able to create both dag_id and task_id with non ascii char:
   ```
   from airflow import DAG
   from datetime import datetime
   from airflow.operators.bash import BashOperator
   
   
   default_args = {
       'owner': 'airflow',
       'start_date': datetime(2020, 3, 11),
   }
   
   
   with DAG(
       dag_id='simple_non_ascii_ñ_ö',
       default_args=default_args,
       schedule_interval='8/15 * * * *',
   ) as dag:
       BashOperator(
           task_id='sleep_ñ_ö',
           bash_command='sleep 5',
       )
   ```
   
   
   ![Screen Shot 2021-09-03 at 22 56 27](https://user-images.githubusercontent.com/45845474/132059476-2018c0ab-610c-40e3-9bb7-f4f167c54bcc.png)
   
   However I wasn't able to create with `♥O◘♦♥O◘♦`:
   ![Screen Shot 2021-09-03 at 23 42 10](https://user-images.githubusercontent.com/45845474/132063411-f14b84bc-3245-4bee-99e4-0ed1081c0a0e.png)
   
   
   We use `validate_key`:
   
   https://github.com/apache/airflow/blob/867e9305f08bf9580f25430d8b6e84071c59f9e6/airflow/utils/helpers.py#L45-L55
   The regex is:
   https://github.com/apache/airflow/blob/867e9305f08bf9580f25430d8b6e84071c59f9e6/airflow/utils/helpers.py#L37
   
   validation is on both the `dag_id` and `task_id`:
   https://github.com/apache/airflow/blob/867e9305f08bf9580f25430d8b6e84071c59f9e6/airflow/models/dag.py#L328
   
   https://github.com/apache/airflow/blob/867e9305f08bf9580f25430d8b6e84071c59f9e6/airflow/models/baseoperator.py#L538
   
   
   A possible fix is:
   `KEY_REGEX = re.compile(r'^[\w.-]+$', re.ASCII)`
   
   Some examples:
   ```
   import re
   KEY_REGEX = re.compile(r'^[\w.-]+$')
   KEY_REGEX_NEW = re.compile(r'^[\w.-]+$', re.ASCII)
   
   strings = ["ascii_a_b", "ascii_ñ_ö", "♥O◘♦♥O◘♦"]
   
   for string in strings:
       print(f"\nresult for {string}:")
       print(KEY_REGEX.match(string))
       print(KEY_REGEX_NEW.match(string))
   ```
   
   
   gives:
   
   ```
   result for ascii_a_b:
   <_sre.SRE_Match object; span=(0, 9), match='ascii_a_b'>
   <_sre.SRE_Match object; span=(0, 9), match='ascii_a_b'>
   
   result for ascii_ñ_ö:
   <_sre.SRE_Match object; span=(0, 9), match='ascii_ñ_ö'>
   None
   
   result for ♥O◘♦♥O◘♦:
   None
   None
   ```
   
   adding `re.ASCII` will show broken dag as expected:
   ![Screen Shot 2021-09-03 at 23 56 05](https://user-images.githubusercontent.com/45845474/132064588-7f2c8eeb-c23e-4279-aca8-e428ab4150e3.png)
   
   @potiuk WDYT?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org