You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/01/24 02:37:52 UTC
[GitHub] [airflow] microhuang opened a new pull request #21053: Fix: Exception when parsing log #20966
microhuang opened a new pull request #21053:
URL: https://github.com/apache/airflow/pull/21053
When the task outputs some strange characters, decoding will lead to "Unicode decodeerror" error due to inconsistent encoding. We can try-catch it or ignore it by specifying the parameter: errors="backslashreplace" .
e.g.
```
line = ''
for raw_line in iter(self.sub_process.stdout.readline, b''):
try:
line = raw_line.decode(output_encoding).rstrip()
except UnicodeDecodeError as err:
print(err, output_encoding, raw_line)
self.log.info("%s", line)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #21053: Fix: Exception when parsing log #20966
Posted by GitBox <gi...@apache.org>.
eladkal commented on pull request #21053:
URL: https://github.com/apache/airflow/pull/21053#issuecomment-1024642811
Tests is failing:
```
airflow/providers/cncf/kubernetes/utils/pod_manager.py:202: error: Incompatible
types in assignment (expression has type "str", variable has type "bytes")
[assignment]
line = line.decode('utf-8', errors="backslashrepla...
^
airflow/providers/cncf/kubernetes/utils/pod_manager.py:203: error: Argument 1
to "parse_log_line" of "PodManager" has incompatible type "bytes"; expected
"str" [arg-type]
timestamp, message = self.parse_log_line(line)
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #21053: Fix: Exception when parsing log #20966
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #21053:
URL: https://github.com/apache/airflow/pull/21053#issuecomment-1020141324
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #21053: Fix: Exception when parsing log #20966
Posted by GitBox <gi...@apache.org>.
eladkal commented on pull request #21053:
URL: https://github.com/apache/airflow/pull/21053#issuecomment-1039343801
@microhuang can you please fix the test?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] microhuang commented on pull request #21053: Fix: Exception when parsing log #20966
Posted by GitBox <gi...@apache.org>.
microhuang commented on pull request #21053:
URL: https://github.com/apache/airflow/pull/21053#issuecomment-1019744600
> Would it be possible to add a test for this?
Okay, I'll add the test code later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #21053: Fix: Exception when parsing log #20966
Posted by GitBox <gi...@apache.org>.
eladkal commented on pull request #21053:
URL: https://github.com/apache/airflow/pull/21053#issuecomment-1039343801
@microhuang can you please fix the test?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #21053: Fix: Exception when parsing log #20966
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #21053:
URL: https://github.com/apache/airflow/pull/21053#discussion_r795015300
##########
File path: airflow/providers/cncf/kubernetes/utils/pod_manager.py
##########
@@ -199,7 +199,8 @@ def follow_logs(since_time: Optional[DateTime] = None) -> Optional[DateTime]:
),
)
for line in logs:
- timestamp, message = self.parse_log_line(line.decode('utf-8'))
+ line = line.decode('utf-8', errors="backslashreplace")
+ timestamp, message = self.parse_log_line(line)
self.log.info(message)
Review comment:
```python
for raw_line in logs:
line = raw_line.decode('utf-8', errors="backslashreplace")
timestamp, message = self.parse_log_line(line)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21053: Fix: Exception when parsing log #20966
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21053:
URL: https://github.com/apache/airflow/pull/21053#issuecomment-1020139727
Static checks need fixing - I recmmend installing pre-commit and `commit --amend` - you will get them fixed for you automatically.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #21053: Fix: Exception when parsing log #20966
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #21053:
URL: https://github.com/apache/airflow/pull/21053#issuecomment-1019738220
The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on pull request #21053: Fix: Exception when parsing log #20966
Posted by GitBox <gi...@apache.org>.
uranusjr commented on pull request #21053:
URL: https://github.com/apache/airflow/pull/21053#issuecomment-1019738059
Would it be possible to add a test for this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #21053: Fix: Exception when parsing log #20966
Posted by GitBox <gi...@apache.org>.
eladkal commented on pull request #21053:
URL: https://github.com/apache/airflow/pull/21053#issuecomment-1070892984
@microhuang do you wish to finish the work on this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #21053: Fix: Exception when parsing log #20966
Posted by GitBox <gi...@apache.org>.
eladkal commented on pull request #21053:
URL: https://github.com/apache/airflow/pull/21053#issuecomment-1070892984
@microhuang do you wish to finish the work on this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org