You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/08/12 16:00:22 UTC
[GitHub] [airflow] troywinter opened a new pull request #10301: [kubernetes][monitor_pod] decode k8s logs when collecting with airflow logger
troywinter opened a new pull request #10301:
URL: https://github.com/apache/airflow/pull/10301
When running Task with KubernetesPodOperator and collecting logs back to airflow, logs are being collected to airflow logger as byte string. It's not a problem with asiic string, but if it contains unicode string, log file will show unicode byte, not readable at all. So this PR decode to utf-8 first, then log to file.
This change has been running perfectly on our production.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] closed pull request #10301: [kubernetes][monitor_pod] decode k8s logs when collecting with airflow logger
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #10301:
URL: https://github.com/apache/airflow/pull/10301
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #10301: [kubernetes][monitor_pod] decode k8s logs when collecting with airflow logger
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #10301:
URL: https://github.com/apache/airflow/pull/10301#issuecomment-799013678
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #10301: [kubernetes][monitor_pod] decode k8s logs when collecting with airflow logger
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #10301:
URL: https://github.com/apache/airflow/pull/10301#issuecomment-673684296
I think I saw at least in one place in the code where we had to do a conditional - as sometimes bytes and sometimes stirngs were returned. I am not sure if it was not because of python2 vs python3 - I think not though.
It was likely kubernetes API issue. @dimberman do you recall that ?
Maybe just for safety we could do something like:
Python 3:
```
[x.encode('utf-8') if isinstance(x, str) else x for x in items]
```
But then if we cherry-pick it to 1.10 it should become:
```
[x.encode('utf-8') if isinstance(x, future.text_type) else x for x in items]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] RosterIn commented on pull request #10301: [kubernetes][monitor_pod] decode k8s logs when collecting with airflow logger
Posted by GitBox <gi...@apache.org>.
RosterIn commented on pull request #10301:
URL: https://github.com/apache/airflow/pull/10301#issuecomment-706508293
not exactly the same but this [article]( https://symantec-enterprise-blogs.security.com/blogs/expert-perspectives/application-logging-kubernetes) explains the need of decode to utf-8
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] stale[bot] commented on pull request #10301: [kubernetes][monitor_pod] decode k8s logs when collecting with airflow logger
Posted by GitBox <gi...@apache.org>.
stale[bot] commented on pull request #10301:
URL: https://github.com/apache/airflow/pull/10301#issuecomment-703189588
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org