You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/01/10 00:20:58 UTC
[GitHub] [airflow] rconroy293 opened a new pull request #7120:
[AIRFLOW-6522] Clear log file to fix duplication in S3TaskHandler
rconroy293 opened a new pull request #7120: [AIRFLOW-6522] Clear log file to fix duplication in S3TaskHandler
URL: https://github.com/apache/airflow/pull/7120
The same task instance (including try number) can be run on a worker
when using a sensor in "reschedule" mode. Accordingly, this clears the
local log file when re-initializing the logger so that the old log
lines aren't uploaded again when the logger is closed.
---
Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg)
- [ ] Description above provides context of the change
- [ ] Commit message/PR title starts with `[AIRFLOW-NNNN]`. AIRFLOW-NNNN = JIRA ID<sup>*</sup>
- [ ] Unit tests coverage for changes (not needed for documentation changes)
- [ ] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
- [ ] Relevant documentation is updated including usage instructions.
- [ ] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
<sup>*</sup> For document-only changes commit message can start with `[AIRFLOW-XXXX]`.
---
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] rconroy293 commented on issue #7120: [AIRFLOW-6522] Clear
log file to fix duplication in S3TaskHandler
Posted by GitBox <gi...@apache.org>.
rconroy293 commented on issue #7120: [AIRFLOW-6522] Clear log file to fix duplication in S3TaskHandler
URL: https://github.com/apache/airflow/pull/7120#issuecomment-573741775
> Trying to think if there are any edge cases to this (such as cases where the upload fails and we might loose the log?)
>
> Can't think of any immediate ones, so LGTM.
The other option I considered was downloading the existing log from S3 into the file first (i.e. on initialization of the logger). If that succeeds, it can just overwrite with the local file. If it fails, it can use the existing append technique. Not sure your thoughts on that @ashb
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] boring-cyborg[bot] commented on issue #7120:
[AIRFLOW-6522] Clear log file to fix duplication in S3TaskHandler
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #7120: [AIRFLOW-6522] Clear log file to fix duplication in S3TaskHandler
URL: https://github.com/apache/airflow/pull/7120#issuecomment-574177120
Awesome work, congrats on your first merged pull request!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ashb merged pull request #7120: [AIRFLOW-6522] Clear log
file to fix duplication in S3TaskHandler
Posted by GitBox <gi...@apache.org>.
ashb merged pull request #7120: [AIRFLOW-6522] Clear log file to fix duplication in S3TaskHandler
URL: https://github.com/apache/airflow/pull/7120
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services