You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/02/13 13:46:53 UTC

[GitHub] [airflow] mik-laj opened a new pull request #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests

mik-laj opened a new pull request #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests
URL: https://github.com/apache/airflow/pull/7412
 
 
   Hello,
   
   I fixed small inconsistencies and added tests to avoid regression.
   
   Best regards,
   Kamil
   
   @feluelle  I also renamed aws_hook to base_aws. WDYT? In separate PR. I will rename airflow.providers.google.cloud.base to airflow.providers.google.common.base_google to unify names.
   
   ---
   Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [ ] Description above provides context of the change
   - [ ] Commit message/PR title starts with `[AIRFLOW-NNNN]`. AIRFLOW-NNNN = JIRA ID<sup>*</sup>
   - [ ] Unit tests coverage for changes (not needed for documentation changes)
   - [ ] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
   - [ ] Relevant documentation is updated including usage instructions.
   - [ ] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   <sup>*</sup> For document-only changes commit message can start with `[AIRFLOW-XXXX]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on issue #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests

Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests
URL: https://github.com/apache/airflow/pull/7412#issuecomment-587078917
 
 
   > ..but why not just `aws / google ? 
   This is too confusing because we also have packages that have the same name.
   
   >  If we want to prefix it with base_ I would suggest to rename the class name also to BaseAwsHook. WDYT?
   We currently have CloudBaseHook class, so maybe let's call AwsBaseHook and GoogleBaseHook. WDYT?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj merged pull request #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests

Posted by GitBox <gi...@apache.org>.
mik-laj merged pull request #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests
URL: https://github.com/apache/airflow/pull/7412
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] feluelle commented on issue #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests

Posted by GitBox <gi...@apache.org>.
feluelle commented on issue #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests
URL: https://github.com/apache/airflow/pull/7412#issuecomment-587083854
 
 
   > We currently have CloudBaseHook class, so maybe let's call AwsBaseHook and GoogleBaseHook. WDYT?
   
   I am fine with this πŸ‘ 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj edited a comment on issue #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests

Posted by GitBox <gi...@apache.org>.
mik-laj edited a comment on issue #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests
URL: https://github.com/apache/airflow/pull/7412#issuecomment-587078917
 
 
   > ..but why not just `aws / google ? 
   
   This is too confusing because we also have packages that have the same name.
   
   >  If we want to prefix it with base_ I would suggest to rename the class name also to BaseAwsHook. WDYT?
   
   We currently have CloudBaseHook class, so maybe let's call AwsBaseHook and GoogleBaseHook. WDYT?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests

Posted by GitBox <gi...@apache.org>.
codecov-io commented on issue #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests
URL: https://github.com/apache/airflow/pull/7412#issuecomment-587207652
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=h1) Report
   > Merging [#7412](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/5b199cb86be5b1aefbd8620185033d6f635713c1?src=pr&el=desc) will **decrease** coverage by `0.38%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7412/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #7412      +/-   ##
   ==========================================
   - Coverage   86.66%   86.27%   -0.39%     
   ==========================================
     Files         878      877       -1     
     Lines       41219    41221       +2     
   ==========================================
   - Hits        35722    35564     -158     
   - Misses       5497     5657     +160
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=tree) | Coverage Ξ” | |
   |---|---|---|
   | [...ow/providers/docker/example\_dags/example\_docker.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZG9ja2VyL2V4YW1wbGVfZGFncy9leGFtcGxlX2RvY2tlci5weQ==) | `100% <ΓΈ> (ΓΈ)` | |
   | [...w/providers/apache/pig/example\_dags/example\_pig.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL3BpZy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9waWcucHk=) | `100% <ΓΈ> (ΓΈ)` | |
   | [...viders/docker/example\_dags/example\_docker\_swarm.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZG9ja2VyL2V4YW1wbGVfZGFncy9leGFtcGxlX2RvY2tlcl9zd2FybS5weQ==) | `100% <ΓΈ> (ΓΈ)` | |
   | [.../providers/google/suite/operators/gcs\_to\_gdrive.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL3N1aXRlL29wZXJhdG9ycy9nY3NfdG9fZ2RyaXZlLnB5) | `100% <ΓΈ> (ΓΈ)` | |
   | [...irflow/providers/http/example\_dags/example\_http.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvaHR0cC9leGFtcGxlX2RhZ3MvZXhhbXBsZV9odHRwLnB5) | `100% <ΓΈ> (ΓΈ)` | |
   | [...cncf/kubernetes/example\_dags/example\_kubernetes.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL2V4YW1wbGVfZGFncy9leGFtcGxlX2t1YmVybmV0ZXMucHk=) | `78.57% <ΓΈ> (ΓΈ)` | |
   | [...providers/google/cloud/example\_dags/example\_dlp.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL2V4YW1wbGVfZGFncy9leGFtcGxlX2RscC5weQ==) | `100% <ΓΈ> (ΓΈ)` | |
   | [...low/providers/yandex/hooks/yandexcloud\_dataproc.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMveWFuZGV4L2hvb2tzL3lhbmRleGNsb3VkX2RhdGFwcm9jLnB5) | `100% <ΓΈ> (ΓΈ)` | |
   | [.../cloud/operators/cloud\_storage\_transfer\_service.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL29wZXJhdG9ycy9jbG91ZF9zdG9yYWdlX3RyYW5zZmVyX3NlcnZpY2UucHk=) | `95.63% <100%> (ΓΈ)` | :arrow_up: |
   | [...flow/providers/amazon/aws/hooks/lambda\_function.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9ob29rcy9sYW1iZGFfZnVuY3Rpb24ucHk=) | `100% <100%> (ΓΈ)` | :arrow_up: |
   | ... and [43 more](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Ξ” = absolute <relative> (impact)`, `ΓΈ = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=footer). Last update [5b199cb...8344222](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7412: [AIRFLOW-6792] Remove _operator/_hook/_sensor in providers package and add tests
URL: https://github.com/apache/airflow/pull/7412#issuecomment-587207652
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=h1) Report
   > Merging [#7412](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/5b199cb86be5b1aefbd8620185033d6f635713c1?src=pr&el=desc) will **decrease** coverage by `0.29%`.
   > The diff coverage is `100%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7412/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff            @@
   ##           master    #7412     +/-   ##
   =========================================
   - Coverage   86.66%   86.37%   -0.3%     
   =========================================
     Files         878      877      -1     
     Lines       41219    41221      +2     
   =========================================
   - Hits        35722    35603    -119     
   - Misses       5497     5618    +121
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=tree) | Coverage Ξ” | |
   |---|---|---|
   | [...ow/providers/docker/example\_dags/example\_docker.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZG9ja2VyL2V4YW1wbGVfZGFncy9leGFtcGxlX2RvY2tlci5weQ==) | `100% <ΓΈ> (ΓΈ)` | |
   | [...w/providers/apache/pig/example\_dags/example\_pig.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL3BpZy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9waWcucHk=) | `100% <ΓΈ> (ΓΈ)` | |
   | [...viders/docker/example\_dags/example\_docker\_swarm.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZG9ja2VyL2V4YW1wbGVfZGFncy9leGFtcGxlX2RvY2tlcl9zd2FybS5weQ==) | `100% <ΓΈ> (ΓΈ)` | |
   | [.../providers/google/suite/operators/gcs\_to\_gdrive.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL3N1aXRlL29wZXJhdG9ycy9nY3NfdG9fZ2RyaXZlLnB5) | `100% <ΓΈ> (ΓΈ)` | |
   | [...irflow/providers/http/example\_dags/example\_http.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvaHR0cC9leGFtcGxlX2RhZ3MvZXhhbXBsZV9odHRwLnB5) | `100% <ΓΈ> (ΓΈ)` | |
   | [...cncf/kubernetes/example\_dags/example\_kubernetes.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL2V4YW1wbGVfZGFncy9leGFtcGxlX2t1YmVybmV0ZXMucHk=) | `78.57% <ΓΈ> (ΓΈ)` | |
   | [...providers/google/cloud/example\_dags/example\_dlp.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL2V4YW1wbGVfZGFncy9leGFtcGxlX2RscC5weQ==) | `100% <ΓΈ> (ΓΈ)` | |
   | [...low/providers/yandex/hooks/yandexcloud\_dataproc.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMveWFuZGV4L2hvb2tzL3lhbmRleGNsb3VkX2RhdGFwcm9jLnB5) | `100% <ΓΈ> (ΓΈ)` | |
   | [.../cloud/operators/cloud\_storage\_transfer\_service.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL29wZXJhdG9ycy9jbG91ZF9zdG9yYWdlX3RyYW5zZmVyX3NlcnZpY2UucHk=) | `95.63% <100%> (ΓΈ)` | :arrow_up: |
   | [...flow/providers/amazon/aws/hooks/lambda\_function.py](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9ob29rcy9sYW1iZGFfZnVuY3Rpb24ucHk=) | `100% <100%> (ΓΈ)` | :arrow_up: |
   | ... and [43 more](https://codecov.io/gh/apache/airflow/pull/7412/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Ξ” = absolute <relative> (impact)`, `ΓΈ = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=footer). Last update [5b199cb...8344222](https://codecov.io/gh/apache/airflow/pull/7412?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services