You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/01/18 14:57:38 UTC

[GitHub] [airflow] nuclearpinguin opened a new pull request #7205: [AIRFLOW-XXXX] Adjust celery defaults to work with breeze

nuclearpinguin opened a new pull request #7205: [AIRFLOW-XXXX] Adjust celery defaults to work with breeze
URL: https://github.com/apache/airflow/pull/7205
 
 
   This PR changes default Celery configs values to ease working with CeleryExecutor in Breeze.
   
   ---
   Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [ ] Description above provides context of the change
   - [ ] Commit message/PR title starts with `[AIRFLOW-NNNN]`. AIRFLOW-NNNN = JIRA ID<sup>*</sup>
   - [ ] Unit tests coverage for changes (not needed for documentation changes)
   - [ ] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
   - [ ] Relevant documentation is updated including usage instructions.
   - [ ] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   <sup>*</sup> For document-only changes commit message can start with `[AIRFLOW-XXXX]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7205: [AIRFLOW-XXXX] Adjust celery defaults to work with breeze

Posted by GitBox <gi...@apache.org>.

nuclearpinguin commented on a change in pull request #7205: [AIRFLOW-XXXX] Adjust celery defaults to work with breeze
URL: https://github.com/apache/airflow/pull/7205#discussion_r368231017
 
 

 ##########
 File path: airflow/config_templates/default_celery.py
 ##########
 @@ -40,7 +40,7 @@ def _broker_supports_visibility_timeout(url):
         broker_transport_options['visibility_timeout'] = 21600
 
 DEFAULT_CELERY_CONFIG = {
-    'accept_content': ['json', 'pickle'],
+    'accept_content': ['json'],
 
 Review comment:
   Reason for this changes:
   ```
   Running a worker with superuser privileges when the
   worker accepts messages serialized with pickle is a very bad idea!
   
   If you really want to continue then you have to set the C_FORCE_ROOT
   environment variable (but please think about this before you do).
   
   User information: uid=0 euid=0 gid=0 egid=0
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk merged pull request #7205: [AIRFLOW-XXXX] Adjust celery defaults to work with breeze

Posted by GitBox <gi...@apache.org>.

potiuk merged pull request #7205: [AIRFLOW-XXXX] Adjust celery defaults to work with breeze
URL: https://github.com/apache/airflow/pull/7205
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #7205: [AIRFLOW-XXXX] Adjust celery defaults to work with breeze

Posted by GitBox <gi...@apache.org>.

codecov-io commented on issue #7205: [AIRFLOW-XXXX] Adjust celery defaults to work with breeze
URL: https://github.com/apache/airflow/pull/7205#issuecomment-575909802
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7205?src=pr&el=h1) Report
   > Merging [#7205](https://codecov.io/gh/apache/airflow/pull/7205?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/417feda5eba6cc63c01a9a04a7b48bc24f63a3d1?src=pr&el=desc) will **decrease** coverage by `0.25%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7205/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7205?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #7205      +/-   ##
   ==========================================
   - Coverage   85.37%   85.12%   -0.26%     
   ==========================================
     Files         723      753      +30     
     Lines       39543    39709     +166     
   ==========================================
   + Hits        33761    33801      +40     
   - Misses       5782     5908     +126
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7205?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [airflow/config\_templates/default\_celery.py](https://codecov.io/gh/apache/airflow/pull/7205/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWdfdGVtcGxhdGVzL2RlZmF1bHRfY2VsZXJ5LnB5) | `52.94% <ø> (ø)` | :arrow_up: |
   | [...rflow/contrib/sensors/sagemaker\_training\_sensor.py](https://codecov.io/gh/apache/airflow/pull/7205/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL3NlbnNvcnMvc2FnZW1ha2VyX3RyYWluaW5nX3NlbnNvci5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
   | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7205/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
   | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7205/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
   | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7205/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
   | [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7205/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
   | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/7205/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `76.47% <0%> (-22.33%)` | :arrow_down: |
   | [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/7205/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `90.43% <0%> (-1.45%)` | :arrow_down: |
   | [airflow/cli/commands/dag\_command.py](https://codecov.io/gh/apache/airflow/pull/7205/diff?src=pr&el=tree#diff-YWlyZmxvdy9jbGkvY29tbWFuZHMvZGFnX2NvbW1hbmQucHk=) | `85.9% <0%> (-0.31%)` | :arrow_down: |
   | [airflow/jobs/scheduler\_job.py](https://codecov.io/gh/apache/airflow/pull/7205/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL3NjaGVkdWxlcl9qb2IucHk=) | `89.23% <0%> (-0.12%)` | :arrow_down: |
   | ... and [63 more](https://codecov.io/gh/apache/airflow/pull/7205/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7205?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7205?src=pr&el=footer). Last update [417feda...e91ecbe](https://codecov.io/gh/apache/airflow/pull/7205?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services