You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/01/10 00:20:58 UTC

[GitHub] [airflow] rconroy293 opened a new pull request #7120: [AIRFLOW-6522] Clear log file to fix duplication in S3TaskHandler

rconroy293 opened a new pull request #7120: [AIRFLOW-6522] Clear log file to fix duplication in S3TaskHandler
URL: https://github.com/apache/airflow/pull/7120
 
 
   The same task instance (including try number) can be run on a worker
   when using a sensor in "reschedule" mode. Accordingly, this clears the
   local log file when re-initializing the logger so that the old log
   lines aren't uploaded again when the logger is closed.
   
   ---
   Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   - [ ] Description above provides context of the change
   - [ ] Commit message/PR title starts with `[AIRFLOW-NNNN]`. AIRFLOW-NNNN = JIRA ID<sup>*</sup>
   - [ ] Unit tests coverage for changes (not needed for documentation changes)
   - [ ] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
   - [ ] Relevant documentation is updated including usage instructions.
   - [ ] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   <sup>*</sup> For document-only changes commit message can start with `[AIRFLOW-XXXX]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] rconroy293 commented on issue #7120: [AIRFLOW-6522] Clear log file to fix duplication in S3TaskHandler

Posted by GitBox <gi...@apache.org>.
rconroy293 commented on issue #7120: [AIRFLOW-6522] Clear log file to fix duplication in S3TaskHandler
URL: https://github.com/apache/airflow/pull/7120#issuecomment-573741775
 
 
   > Trying to think if there are any edge cases to this (such as cases where the upload fails and we might loose the log?)
   > 
   > Can't think of any immediate ones, so LGTM.
   
   The other option I considered was downloading the existing log from S3 into the file first (i.e. on initialization of the logger). If that succeeds, it can just overwrite with the local file. If it fails, it can use the existing append technique. Not sure your thoughts on that @ashb 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] boring-cyborg[bot] commented on issue #7120: [AIRFLOW-6522] Clear log file to fix duplication in S3TaskHandler

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #7120: [AIRFLOW-6522] Clear log file to fix duplication in S3TaskHandler
URL: https://github.com/apache/airflow/pull/7120#issuecomment-574177120
 
 
   Awesome work, congrats on your first merged pull request!
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] ashb merged pull request #7120: [AIRFLOW-6522] Clear log file to fix duplication in S3TaskHandler

Posted by GitBox <gi...@apache.org>.
ashb merged pull request #7120: [AIRFLOW-6522] Clear log file to fix duplication in S3TaskHandler
URL: https://github.com/apache/airflow/pull/7120
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services