You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "rawwar (via GitHub)" <gi...@apache.org> on 2024/05/02 11:32:30 UTC

[PR] [feat] Add pattern matching to `SafeDogStatsdLogger` [airflow]

rawwar opened a new pull request, #39370:
URL: https://github.com/apache/airflow/pull/39370

   closes #39368 
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Refactor `SafeDogStatsdLogger` to use `get_validator` to enable pattern matching [airflow]

Posted by "pankajkoti (via GitHub)" <gi...@apache.org>.
pankajkoti merged PR #39370:
URL: https://github.com/apache/airflow/pull/39370


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Refactor `SafeDogStatsdLogger` to use `get_validator` to enable pattern matching [airflow]

Posted by "rawwar (via GitHub)" <gi...@apache.org>.
rawwar commented on PR #39370:
URL: https://github.com/apache/airflow/pull/39370#issuecomment-2092397854

   @pankajkoti, Yes, I have tested them. Allow/Block list without pattern worked perfectly.  However, I noticed the following behaviour when using pattern matching, which I am not sure if it's right.
   
   ## With pattern matching enabled:
   
   1. `AIRFLOW__METRICS__METRICS_ALLOW_LIST` =  `.*queued_duration.*` : Worked perfectly
   
   
   <img width="925" alt="image" src="https://github.com/apache/airflow/assets/20266953/0b380cca-2a2b-4795-9af2-0af864cacda9">
   
   
   2. `AIRFLOW__METRICS__METRICS_ALLOW_LIST` =  `.*get_astronauts.*` : Not sure if this is correct
   
   
   <img width="910" alt="image" src="https://github.com/apache/airflow/assets/20266953/8aeda08c-156b-488f-a5ff-c4467f8a34ae">
   
   
   
   In the 2nd screenshot, I still see several metrics that don't have `get_astronauts` anywhere in their metrics names. Is this correct behaviour? One of them is `airflow.ti.finish`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Refactor `SafeDogStatsdLogger` to use `get_validator` to enable pattern matching [airflow]

Posted by "rawwar (via GitHub)" <gi...@apache.org>.
rawwar commented on PR #39370:
URL: https://github.com/apache/airflow/pull/39370#issuecomment-2092410935

   Block lists seem to be working as intended.
   
   1. `AIRFLOW__METRICS__METRICS_BLOCK_LIST` = `.*duration.*` : works perfectly
   
   <img width="1257" alt="image" src="https://github.com/apache/airflow/assets/20266953/29dc1103-778d-4b6d-b69e-2a4938bb6180">
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Refactor `SafeDogStatsdLogger` to use `get_validator` to enable pattern matching [airflow]

Posted by "rawwar (via GitHub)" <gi...@apache.org>.
rawwar commented on PR #39370:
URL: https://github.com/apache/airflow/pull/39370#issuecomment-2092840761

   Env variables set:
   
   
   AIRFLOW__METRICS__STATSD_DATADOG_ENABLED | true 
   -- | -- | --
   AIRFLOW__METRICS__STATSD_DATADOG_TAGS | testNo:C2 
   DATADOG_API_KEY | ● ● ● ● ● ● ●
   AIRFLOW__METRICS__METRICS_USE_PATTERN_MATCH | true
   AIRFLOW__METRICS__METRICS_ALLOW_LIST | .*print_astronaut_craft.*
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Refactor `SafeDogStatsdLogger` to use `get_validator` to enable pattern matching [airflow]

Posted by "rawwar (via GitHub)" <gi...@apache.org>.
rawwar commented on PR #39370:
URL: https://github.com/apache/airflow/pull/39370#issuecomment-2093913390

   > Code change LGTM, but I'm not sure that second allow-list example (the one you seemed unsure about) is working as expected. Either way, that is not in scope for this PR if that is broken, but you may want to look into that more if you have the time.
   
   
   I will look into it. I will create an issue and post my findings there.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org