You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/01/29 07:39:33 UTC
[GitHub] [airflow] uranusjr commented on a change in pull request #21175: Fixes Docker xcom functionality
uranusjr commented on a change in pull request #21175:
URL: https://github.com/apache/airflow/pull/21175#discussion_r795019262
##########
File path: airflow/providers/docker/operators/docker.py
##########
@@ -275,32 +275,40 @@ def _run_image_with_mounts(self, target_mounts, add_tmp_variable: bool) -> Optio
working_dir=self.working_dir,
tty=self.tty,
)
- lines = self.cli.attach(container=self.container['Id'], stdout=True, stderr=True, stream=True)
+ logstream = self.cli.attach(container=self.container['Id'], stdout=True, stderr=True, stream=True)
try:
self.cli.start(self.container['Id'])
- line = ''
- res_lines = []
- return_value = None
- for line in lines:
- if hasattr(line, 'decode'):
+ log_lines = []
+ for log_chunk in logstream:
+ if hasattr(log_chunk, 'decode'):
# Note that lines returned can also be byte sequences so we have to handle decode here
- line = line.decode('utf-8')
- line = line.strip()
- res_lines.append(line)
- self.log.info(line)
+ log_chunk = log_chunk.decode('utf-8')
Review comment:
We probably should have `error="surrogateescape"` here (something like this, I don’t remember the exact name) because there’s no guaratee the log is outputting valid strings, and those characters shouldn’t crash the entire job.
##########
File path: airflow/providers/docker/operators/docker.py
##########
@@ -275,32 +275,40 @@ def _run_image_with_mounts(self, target_mounts, add_tmp_variable: bool) -> Optio
working_dir=self.working_dir,
tty=self.tty,
)
- lines = self.cli.attach(container=self.container['Id'], stdout=True, stderr=True, stream=True)
+ logstream = self.cli.attach(container=self.container['Id'], stdout=True, stderr=True, stream=True)
try:
self.cli.start(self.container['Id'])
- line = ''
- res_lines = []
- return_value = None
- for line in lines:
- if hasattr(line, 'decode'):
+ log_lines = []
+ for log_chunk in logstream:
+ if hasattr(log_chunk, 'decode'):
# Note that lines returned can also be byte sequences so we have to handle decode here
- line = line.decode('utf-8')
- line = line.strip()
- res_lines.append(line)
- self.log.info(line)
+ log_chunk = log_chunk.decode('utf-8')
+ log_chunk = log_chunk.strip()
+ log_lines.append(log_chunk)
+ self.log.info(log_chunk)
Review comment:
```suggestion
self.log.info("%s", log_chunk)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org