You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "dstandish (via GitHub)" <gi...@apache.org> on 2023/05/09 23:14:14 UTC

[GitHub] [airflow] dstandish opened a new pull request, #31175: Fix timestamp parse failure for k8s executor pod tailing

dstandish opened a new pull request, #31175:
URL: https://github.com/apache/airflow/pull/31175

   When we tail stdout over kube API, we get ansi color codes which must be escaped in order for timestamps to be properly parsed so that messages can be properly interleaved by the file task handler (i.e. / a.k.a. the web UI task log reader).
   
   While at it, fixed an issue with improper appending of "messages" (e.g. "fetching logs from kube API") and added one such message for good measure.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] dstandish commented on a diff in pull request #31175: Fix timestamp parse failure for k8s executor pod tailing

Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on code in PR #31175:
URL: https://github.com/apache/airflow/pull/31175#discussion_r1190243810


##########
airflow/executors/kubernetes_executor.py:
##########
@@ -820,9 +821,11 @@ def get_task_log(self, ti: TaskInstance, try_number: int) -> tuple[list[str], li
                 tail_lines=100,
                 _preload_content=False,
             )
-
+            ansi_escape = re.compile(r"\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])")

Review Comment:
   updated to use existing function



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] dstandish commented on a diff in pull request #31175: Fix timestamp parse failure for k8s executor pod tailing

Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on code in PR #31175:
URL: https://github.com/apache/airflow/pull/31175#discussion_r1190240422


##########
airflow/executors/kubernetes_executor.py:
##########
@@ -820,9 +821,11 @@ def get_task_log(self, ti: TaskInstance, try_number: int) -> tuple[list[str], li
                 tail_lines=100,
                 _preload_content=False,
             )
-
+            ansi_escape = re.compile(r"\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])")

Review Comment:
   ah HAH we already have this in the repo :) 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] dstandish commented on a diff in pull request #31175: Fix timestamp parse failure for k8s executor pod tailing

Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on code in PR #31175:
URL: https://github.com/apache/airflow/pull/31175#discussion_r1190237400


##########
airflow/executors/kubernetes_executor.py:
##########
@@ -820,9 +821,11 @@ def get_task_log(self, ti: TaskInstance, try_number: int) -> tuple[list[str], li
                 tail_lines=100,
                 _preload_content=False,
             )
-
+            ansi_escape = re.compile(r"\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])")

Review Comment:
   i agree... but i thought that the variable name was sufficient:  `ansi_escape` 
   
   wdyt?
   
   there's a more verbose form:
   
   ```
   import re
   
   # 7-bit C1 ANSI sequences
   ansi_escape = re.compile(r'''
       \x1B  # ESC
       (?:   # 7-bit C1 Fe (except CSI)
           [@-Z\\-_]
       |     # or [ for CSI, followed by a control sequence
           \[
           [0-?]*  # Parameter bytes
           [ -/]*  # Intermediate bytes
           [@-~]   # Final byte
       )
   ''', re.VERBOSE)
   ```
   
   All blatantly stolen from stack overflow https://stackoverflow.com/a/14693789/2761682
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] uranusjr commented on a diff in pull request #31175: Fix timestamp parse failure for k8s executor pod tailing

Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #31175:
URL: https://github.com/apache/airflow/pull/31175#discussion_r1189431596


##########
airflow/executors/kubernetes_executor.py:
##########
@@ -820,9 +821,11 @@ def get_task_log(self, ti: TaskInstance, try_number: int) -> tuple[list[str], li
                 tail_lines=100,
                 _preload_content=False,
             )
-
+            ansi_escape = re.compile(r"\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])")

Review Comment:
   Some comments would be helpful, everyone hates needing to read regex to understand what it does!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] uranusjr merged pull request #31175: Fix timestamp parse failure for k8s executor pod tailing

Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr merged PR #31175:
URL: https://github.com/apache/airflow/pull/31175


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org