You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/02/02 23:23:10 UTC
[GitHub] [airflow] alex-astronomer opened a new pull request #21281: Mask secrets in stdout for `airflow tasks test ...` CLI command (#17476)
alex-astronomer opened a new pull request #21281:
URL: https://github.com/apache/airflow/pull/21281
The problem with `airflow tasks test ...` is that any `stdout` that comes from a task inside an operator or other callable does not go through a logger. It runs in the main process. This results in secrets that are printed to `stdout` to not be masked according to the `airflow.task` logger.
I wanted a way to capture and filter stdout without having to route everything through a secondary logger. I decided a context manager that filters `stdout` would be the best way to go about this. The context manager redacts all secrets according to the filters on the `airflow.task` logger.
Definitely not sure if this is the right way to do it but it was the best solution that I could come up with!
Closes #17476
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] alex-astronomer commented on a change in pull request #21281: Mask secrets in stdout for `airflow tasks test ...` CLI command (#17476)
Posted by GitBox <gi...@apache.org>.
alex-astronomer commented on a change in pull request #21281:
URL: https://github.com/apache/airflow/pull/21281#discussion_r800104242
##########
File path: airflow/utils/log/secrets_masker.py
##########
@@ -252,3 +253,44 @@ def add_mask(self, secret: Union[str, dict, Iterable], name: Optional[str] = Non
elif isinstance(secret, collections.abc.Iterable):
for v in secret:
self.add_mask(v, name)
+
+
+class StdoutRedactContext:
+ """
+ This class redirects all stdout within the context through the logger specified in the init.
+ Print statements get turned into log lines through the logger member.
+
+ """
+
+ def __init__(self):
+ self.stdout_lines = []
+ self._redirector = contextlib.redirect_stdout(self)
+
+ def write(self, stdout_line):
+ """Write stdout to the internal buffer when stdout is written to."""
+ self.stdout_lines.append(redact(stdout_line))
+
+ def flush(self):
+ """
+ No flush functionality needed here, but this method is needed to make this class a "file-like" which
+ can be called by the context lib in self._redirector.
+
+ """
+
+ def __enter__(self):
+ """
+ At the beginning of the context, enter the "redirect mode" which is initiated by entering the context
+ of `contextlib.redirect_stdout`.
+
+ """
+ self._redirector.__enter__()
+
+ def __exit__(self, *args):
+ """
+ On exit from the context, close the `contextlib.redirect_stdout` context to stop capturing lines
+ from stdout, then route all lines one by one through self.redirect_to.
+
+ """
+ self._redirector.__exit__(*args)
+ for log in self.stdout_lines:
+ sys.stdout.write(log)
Review comment:
Great find! I'll implement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #21281: Mask secrets in stdout for `airflow tasks test ...` CLI command (#17476)
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #21281:
URL: https://github.com/apache/airflow/pull/21281#discussion_r799136303
##########
File path: airflow/utils/log/secrets_masker.py
##########
@@ -252,3 +253,44 @@ def add_mask(self, secret: Union[str, dict, Iterable], name: Optional[str] = Non
elif isinstance(secret, collections.abc.Iterable):
for v in secret:
self.add_mask(v, name)
+
+
+class StdoutRedactContext:
+ """
+ This class redirects all stdout within the context through the logger specified in the init.
+ Print statements get turned into log lines through the logger member.
+
+ """
+
+ def __init__(self):
+ self.stdout_lines = []
+ self._redirector = contextlib.redirect_stdout(self)
+
+ def write(self, stdout_line):
+ """Write stdout to the internal buffer when stdout is written to."""
+ self.stdout_lines.append(redact(stdout_line))
+
+ def flush(self):
+ """
+ No flush functionality needed here, but this method is needed to make this class a "file-like" which
+ can be called by the context lib in self._redirector.
+
+ """
+
+ def __enter__(self):
+ """
+ At the beginning of the context, enter the "redirect mode" which is initiated by entering the context
+ of `contextlib.redirect_stdout`.
+
+ """
+ self._redirector.__enter__()
+
+ def __exit__(self, *args):
+ """
+ On exit from the context, close the `contextlib.redirect_stdout` context to stop capturing lines
+ from stdout, then route all lines one by one through self.redirect_to.
+
+ """
+ self._redirector.__exit__(*args)
+ for log in self.stdout_lines:
+ sys.stdout.write(log)
Review comment:
> it’ll get stuck in an infinite recursive loop
Ah right. `sys.__stdout__` then?
##########
File path: airflow/utils/log/secrets_masker.py
##########
@@ -252,3 +253,44 @@ def add_mask(self, secret: Union[str, dict, Iterable], name: Optional[str] = Non
elif isinstance(secret, collections.abc.Iterable):
for v in secret:
self.add_mask(v, name)
+
+
+class StdoutRedactContext:
+ """
+ This class redirects all stdout within the context through the logger specified in the init.
+ Print statements get turned into log lines through the logger member.
+
+ """
+
+ def __init__(self):
+ self.stdout_lines = []
+ self._redirector = contextlib.redirect_stdout(self)
+
+ def write(self, stdout_line):
+ """Write stdout to the internal buffer when stdout is written to."""
+ self.stdout_lines.append(redact(stdout_line))
+
+ def flush(self):
+ """
+ No flush functionality needed here, but this method is needed to make this class a "file-like" which
+ can be called by the context lib in self._redirector.
+
+ """
+
+ def __enter__(self):
+ """
+ At the beginning of the context, enter the "redirect mode" which is initiated by entering the context
+ of `contextlib.redirect_stdout`.
+
+ """
+ self._redirector.__enter__()
+
+ def __exit__(self, *args):
+ """
+ On exit from the context, close the `contextlib.redirect_stdout` context to stop capturing lines
+ from stdout, then route all lines one by one through self.redirect_to.
+
+ """
+ self._redirector.__exit__(*args)
+ for log in self.stdout_lines:
+ sys.stdout.write(log)
Review comment:
> it’ll get stuck in an infinite recursive loop
Ah right. `sys.__stdout__` then?
https://docs.python.org/3/library/sys.html#sys.__stdout__
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] alex-astronomer commented on a change in pull request #21281: Mask secrets in stdout for `airflow tasks test ...` CLI command (#17476)
Posted by GitBox <gi...@apache.org>.
alex-astronomer commented on a change in pull request #21281:
URL: https://github.com/apache/airflow/pull/21281#discussion_r806075120
##########
File path: airflow/utils/log/secrets_masker.py
##########
@@ -252,3 +253,44 @@ def add_mask(self, secret: Union[str, dict, Iterable], name: Optional[str] = Non
elif isinstance(secret, collections.abc.Iterable):
for v in secret:
self.add_mask(v, name)
+
+
+class StdoutRedactContext:
+ """
+ This class redirects all stdout within the context through the logger specified in the init.
+ Print statements get turned into log lines through the logger member.
+
+ """
+
+ def __init__(self):
+ self.stdout_lines = []
+ self._redirector = contextlib.redirect_stdout(self)
+
+ def write(self, stdout_line):
+ """Write stdout to the internal buffer when stdout is written to."""
+ self.stdout_lines.append(redact(stdout_line))
+
+ def flush(self):
+ """
+ No flush functionality needed here, but this method is needed to make this class a "file-like" which
+ can be called by the context lib in self._redirector.
+
+ """
+
+ def __enter__(self):
+ """
+ At the beginning of the context, enter the "redirect mode" which is initiated by entering the context
+ of `contextlib.redirect_stdout`.
+
+ """
+ self._redirector.__enter__()
+
+ def __exit__(self, *args):
+ """
+ On exit from the context, close the `contextlib.redirect_stdout` context to stop capturing lines
+ from stdout, then route all lines one by one through self.redirect_to.
+
+ """
+ self._redirector.__exit__(*args)
+ for log in self.stdout_lines:
+ sys.stdout.write(log)
Review comment:
So just to make sure that I'm understanding all of this right, we're basically saving a copy of sys.stdout from before it gets redirected, and then writing to that which bypasses the redirect context that we've already defined before that write occurs? And the reason that we use the dataclass decorator is just to simplify the initialization of self.stdout and save a few lines since we don't need to use `__init__`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] alex-astronomer commented on a change in pull request #21281: Mask secrets in stdout for `airflow tasks test ...` CLI command (#17476)
Posted by GitBox <gi...@apache.org>.
alex-astronomer commented on a change in pull request #21281:
URL: https://github.com/apache/airflow/pull/21281#discussion_r798629986
##########
File path: airflow/utils/log/secrets_masker.py
##########
@@ -252,3 +253,44 @@ def add_mask(self, secret: Union[str, dict, Iterable], name: Optional[str] = Non
elif isinstance(secret, collections.abc.Iterable):
for v in secret:
self.add_mask(v, name)
+
+
+class StdoutRedactContext:
+ """
+ This class redirects all stdout within the context through the logger specified in the init.
+ Print statements get turned into log lines through the logger member.
+
+ """
+
+ def __init__(self):
+ self.stdout_lines = []
+ self._redirector = contextlib.redirect_stdout(self)
+
+ def write(self, stdout_line):
+ """Write stdout to the internal buffer when stdout is written to."""
+ self.stdout_lines.append(redact(stdout_line))
+
+ def flush(self):
+ """
+ No flush functionality needed here, but this method is needed to make this class a "file-like" which
+ can be called by the context lib in self._redirector.
+
+ """
+
+ def __enter__(self):
+ """
+ At the beginning of the context, enter the "redirect mode" which is initiated by entering the context
+ of `contextlib.redirect_stdout`.
+
+ """
+ self._redirector.__enter__()
+
+ def __exit__(self, *args):
+ """
+ On exit from the context, close the `contextlib.redirect_stdout` context to stop capturing lines
+ from stdout, then route all lines one by one through self.redirect_to.
+
+ """
+ self._redirector.__exit__(*args)
+ for log in self.stdout_lines:
+ sys.stdout.write(log)
Review comment:
I think the problem here is that every time it writes to `stdout` you’ll need to exit the internal redirect context manager because otherwise it’ll get stuck in an infinite recursive loop. I thought about doing that but since ‘__exit__’ expects a few arguments that only exist during the external context manager wrapper’s exit function. Do you have any thoughts about being able to disable and reenable the internal redirect context manager within the ‘write’ function?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] alex-astronomer commented on a change in pull request #21281: Mask secrets in stdout for `airflow tasks test ...` CLI command (#17476)
Posted by GitBox <gi...@apache.org>.
alex-astronomer commented on a change in pull request #21281:
URL: https://github.com/apache/airflow/pull/21281#discussion_r798629986
##########
File path: airflow/utils/log/secrets_masker.py
##########
@@ -252,3 +253,44 @@ def add_mask(self, secret: Union[str, dict, Iterable], name: Optional[str] = Non
elif isinstance(secret, collections.abc.Iterable):
for v in secret:
self.add_mask(v, name)
+
+
+class StdoutRedactContext:
+ """
+ This class redirects all stdout within the context through the logger specified in the init.
+ Print statements get turned into log lines through the logger member.
+
+ """
+
+ def __init__(self):
+ self.stdout_lines = []
+ self._redirector = contextlib.redirect_stdout(self)
+
+ def write(self, stdout_line):
+ """Write stdout to the internal buffer when stdout is written to."""
+ self.stdout_lines.append(redact(stdout_line))
+
+ def flush(self):
+ """
+ No flush functionality needed here, but this method is needed to make this class a "file-like" which
+ can be called by the context lib in self._redirector.
+
+ """
+
+ def __enter__(self):
+ """
+ At the beginning of the context, enter the "redirect mode" which is initiated by entering the context
+ of `contextlib.redirect_stdout`.
+
+ """
+ self._redirector.__enter__()
+
+ def __exit__(self, *args):
+ """
+ On exit from the context, close the `contextlib.redirect_stdout` context to stop capturing lines
+ from stdout, then route all lines one by one through self.redirect_to.
+
+ """
+ self._redirector.__exit__(*args)
+ for log in self.stdout_lines:
+ sys.stdout.write(log)
Review comment:
I think the problem here is that every time it writes to `stdout` you’ll need to exit the internal redirect context manager because otherwise it’ll get stuck in an infinite recursive loop. I thought about doing that but since `__exit__` expects a few arguments that only exist during the external context manager wrapper’s exit function. Do you have any thoughts about being able to disable and reenable the internal redirect context manager within the ‘write’ function?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] alex-astronomer commented on a change in pull request #21281: Mask secrets in stdout for `airflow tasks test ...` CLI command (#17476)
Posted by GitBox <gi...@apache.org>.
alex-astronomer commented on a change in pull request #21281:
URL: https://github.com/apache/airflow/pull/21281#discussion_r798629986
##########
File path: airflow/utils/log/secrets_masker.py
##########
@@ -252,3 +253,44 @@ def add_mask(self, secret: Union[str, dict, Iterable], name: Optional[str] = Non
elif isinstance(secret, collections.abc.Iterable):
for v in secret:
self.add_mask(v, name)
+
+
+class StdoutRedactContext:
+ """
+ This class redirects all stdout within the context through the logger specified in the init.
+ Print statements get turned into log lines through the logger member.
+
+ """
+
+ def __init__(self):
+ self.stdout_lines = []
+ self._redirector = contextlib.redirect_stdout(self)
+
+ def write(self, stdout_line):
+ """Write stdout to the internal buffer when stdout is written to."""
+ self.stdout_lines.append(redact(stdout_line))
+
+ def flush(self):
+ """
+ No flush functionality needed here, but this method is needed to make this class a "file-like" which
+ can be called by the context lib in self._redirector.
+
+ """
+
+ def __enter__(self):
+ """
+ At the beginning of the context, enter the "redirect mode" which is initiated by entering the context
+ of `contextlib.redirect_stdout`.
+
+ """
+ self._redirector.__enter__()
+
+ def __exit__(self, *args):
+ """
+ On exit from the context, close the `contextlib.redirect_stdout` context to stop capturing lines
+ from stdout, then route all lines one by one through self.redirect_to.
+
+ """
+ self._redirector.__exit__(*args)
+ for log in self.stdout_lines:
+ sys.stdout.write(log)
Review comment:
I think the problem here is that every time it writes to ‘stdout’ you’ll need to exit the internal redirect context manager because otherwise it’ll get stuck in an infinite recursive loop. I thought about doing that but since ‘__exit__’ expects a few arguments that only exist during the external context manager wrapper’s exit function. Do you have any thoughts about being able to disable and reenable the internal redirect context manager within the ‘write’ function?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #21281: Mask secrets in stdout for `airflow tasks test ...` CLI command (#17476)
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #21281:
URL: https://github.com/apache/airflow/pull/21281#discussion_r798211852
##########
File path: airflow/utils/log/secrets_masker.py
##########
@@ -252,3 +253,44 @@ def add_mask(self, secret: Union[str, dict, Iterable], name: Optional[str] = Non
elif isinstance(secret, collections.abc.Iterable):
for v in secret:
self.add_mask(v, name)
+
+
+class StdoutRedactContext:
+ """
+ This class redirects all stdout within the context through the logger specified in the init.
+ Print statements get turned into log lines through the logger member.
+
+ """
+
+ def __init__(self):
+ self.stdout_lines = []
+ self._redirector = contextlib.redirect_stdout(self)
+
+ def write(self, stdout_line):
+ """Write stdout to the internal buffer when stdout is written to."""
+ self.stdout_lines.append(redact(stdout_line))
+
+ def flush(self):
+ """
+ No flush functionality needed here, but this method is needed to make this class a "file-like" which
+ can be called by the context lib in self._redirector.
+
+ """
+
+ def __enter__(self):
+ """
+ At the beginning of the context, enter the "redirect mode" which is initiated by entering the context
+ of `contextlib.redirect_stdout`.
+
+ """
+ self._redirector.__enter__()
+
+ def __exit__(self, *args):
+ """
+ On exit from the context, close the `contextlib.redirect_stdout` context to stop capturing lines
+ from stdout, then route all lines one by one through self.redirect_to.
+
+ """
+ self._redirector.__exit__(*args)
+ for log in self.stdout_lines:
+ sys.stdout.write(log)
Review comment:
Hmm, this means all of the task’s output will be buffered, and only spit out in one go when the task is finished, which seems wrong. I would call `stdout.write` in `write` directly after redaction instead.
This can be significantly simplified to
```python
class StdoutRedactor:
@contextlib.contextmanager
@classmethod
def enable(cls):
with contextlib.redirect_stdout(cls()):
yield
def write(self, content):
sys.stdout.write(redact(content))
def flush(self):
sys.stdout.flush()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] alex-astronomer commented on a change in pull request #21281: Mask secrets in stdout for `airflow tasks test ...` CLI command (#17476)
Posted by GitBox <gi...@apache.org>.
alex-astronomer commented on a change in pull request #21281:
URL: https://github.com/apache/airflow/pull/21281#discussion_r806075120
##########
File path: airflow/utils/log/secrets_masker.py
##########
@@ -252,3 +253,44 @@ def add_mask(self, secret: Union[str, dict, Iterable], name: Optional[str] = Non
elif isinstance(secret, collections.abc.Iterable):
for v in secret:
self.add_mask(v, name)
+
+
+class StdoutRedactContext:
+ """
+ This class redirects all stdout within the context through the logger specified in the init.
+ Print statements get turned into log lines through the logger member.
+
+ """
+
+ def __init__(self):
+ self.stdout_lines = []
+ self._redirector = contextlib.redirect_stdout(self)
+
+ def write(self, stdout_line):
+ """Write stdout to the internal buffer when stdout is written to."""
+ self.stdout_lines.append(redact(stdout_line))
+
+ def flush(self):
+ """
+ No flush functionality needed here, but this method is needed to make this class a "file-like" which
+ can be called by the context lib in self._redirector.
+
+ """
+
+ def __enter__(self):
+ """
+ At the beginning of the context, enter the "redirect mode" which is initiated by entering the context
+ of `contextlib.redirect_stdout`.
+
+ """
+ self._redirector.__enter__()
+
+ def __exit__(self, *args):
+ """
+ On exit from the context, close the `contextlib.redirect_stdout` context to stop capturing lines
+ from stdout, then route all lines one by one through self.redirect_to.
+
+ """
+ self._redirector.__exit__(*args)
+ for log in self.stdout_lines:
+ sys.stdout.write(log)
Review comment:
So just to make sure that I'm understanding all of this right, we're basically saving a copy of sys.stdout from before it gets redirected, and then writing to that which bypasses the redirect context that we've already defined before that write occurs? And the reason that we use the dataclass decorator is just to simplify the initialization of self.stdout and save a few lines since we don't need to use `__init__`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #21281: Mask secrets in stdout for `airflow tasks test ...` CLI command (#17476)
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #21281:
URL: https://github.com/apache/airflow/pull/21281#discussion_r811141939
##########
File path: airflow/utils/log/secrets_masker.py
##########
@@ -252,3 +253,44 @@ def add_mask(self, secret: Union[str, dict, Iterable], name: Optional[str] = Non
elif isinstance(secret, collections.abc.Iterable):
for v in secret:
self.add_mask(v, name)
+
+
+class StdoutRedactContext:
+ """
+ This class redirects all stdout within the context through the logger specified in the init.
+ Print statements get turned into log lines through the logger member.
+
+ """
+
+ def __init__(self):
+ self.stdout_lines = []
+ self._redirector = contextlib.redirect_stdout(self)
+
+ def write(self, stdout_line):
+ """Write stdout to the internal buffer when stdout is written to."""
+ self.stdout_lines.append(redact(stdout_line))
+
+ def flush(self):
+ """
+ No flush functionality needed here, but this method is needed to make this class a "file-like" which
+ can be called by the context lib in self._redirector.
+
+ """
+
+ def __enter__(self):
+ """
+ At the beginning of the context, enter the "redirect mode" which is initiated by entering the context
+ of `contextlib.redirect_stdout`.
+
+ """
+ self._redirector.__enter__()
+
+ def __exit__(self, *args):
+ """
+ On exit from the context, close the `contextlib.redirect_stdout` context to stop capturing lines
+ from stdout, then route all lines one by one through self.redirect_to.
+
+ """
+ self._redirector.__exit__(*args)
+ for log in self.stdout_lines:
+ sys.stdout.write(log)
Review comment:
Yes, that’s my understanding.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #21281: Mask secrets in stdout for `airflow tasks test ...` CLI command (#17476)
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #21281:
URL: https://github.com/apache/airflow/pull/21281#discussion_r800210644
##########
File path: airflow/utils/log/secrets_masker.py
##########
@@ -252,3 +253,44 @@ def add_mask(self, secret: Union[str, dict, Iterable], name: Optional[str] = Non
elif isinstance(secret, collections.abc.Iterable):
for v in secret:
self.add_mask(v, name)
+
+
+class StdoutRedactContext:
+ """
+ This class redirects all stdout within the context through the logger specified in the init.
+ Print statements get turned into log lines through the logger member.
+
+ """
+
+ def __init__(self):
+ self.stdout_lines = []
+ self._redirector = contextlib.redirect_stdout(self)
+
+ def write(self, stdout_line):
+ """Write stdout to the internal buffer when stdout is written to."""
+ self.stdout_lines.append(redact(stdout_line))
+
+ def flush(self):
+ """
+ No flush functionality needed here, but this method is needed to make this class a "file-like" which
+ can be called by the context lib in self._redirector.
+
+ """
+
+ def __enter__(self):
+ """
+ At the beginning of the context, enter the "redirect mode" which is initiated by entering the context
+ of `contextlib.redirect_stdout`.
+
+ """
+ self._redirector.__enter__()
+
+ def __exit__(self, *args):
+ """
+ On exit from the context, close the `contextlib.redirect_stdout` context to stop capturing lines
+ from stdout, then route all lines one by one through self.redirect_to.
+
+ """
+ self._redirector.__exit__(*args)
+ for log in self.stdout_lines:
+ sys.stdout.write(log)
Review comment:
I just realised a problem, `sys.stdout` may have already been patched before we try to patch it (can happen if the user runs Airflow with custom interpreter startup code), in which case `sys.__stdout__` would point to the wrong, unpatched stream.
Something like this is needed:
```python
@dataclasses.dataclass()
class StdoutRedactor:
stdout: TextIO
@contextlib.contextmanager
@classmethod
def enable(cls):
with contextlib.redirect_stdout(cls(sys.stdout)):
yield
def write(self, content):
self.stdout.write(redact(content))
def flush(self):
self.stdout.flush()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org