You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2019/12/25 07:52:25 UTC
[GitHub] [airflow] aviemzur opened a new pull request #6898: fix
EmrAddStepsOperator broken ref & faulty test
aviemzur opened a new pull request #6898: fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [ ] Here are some details about my PR, including screenshots of any UI changes:
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for this extremely good reason:
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes how to use it.
- All the public functions and the classes in the PR contain docstrings that explain what it does
- If you implement backwards incompatible changes, please leave a note in the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so we can assign it to a appropriate release
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test
case for missing cluster
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test case for missing cluster
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/581a39575108e4cfe936f10644c62b1911087948?src=pr&el=desc) will **decrease** coverage by `0.28%`.
> The diff coverage is `87.5%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 84.33% -0.29%
==========================================
Files 679 679
Lines 38505 38509 +4
==========================================
- Hits 32584 32477 -107
- Misses 5921 6032 +111
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.89% <100%> (+0.71%)` | :arrow_up: |
| [airflow/contrib/hooks/emr\_hook.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2Vtcl9ob29rLnB5) | `93.75% <66.66%> (+0.2%)` | :arrow_up: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.75% <0%> (-20%)` | :arrow_down: |
| [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `91.59% <0%> (-0.29%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [581a395...d5d7fc0](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] nuclearpinguin commented on a change in pull request
#6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
Posted by GitBox <gi...@apache.org>.
nuclearpinguin commented on a change in pull request #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898#discussion_r361287794
##########
File path: tests/contrib/operators/test_emr_add_steps_operator.py
##########
@@ -107,23 +107,26 @@ def test_execute_returns_step_id(self):
def test_init_with_cluster_name(self):
expected_job_flow_id = 'j-1231231234'
- self.emr_client_mock.get_cluster_id_by_name.return_value = expected_job_flow_id
self.emr_client_mock.add_job_flow_steps.return_value = ADD_STEPS_SUCCESS_RETURN
with patch('boto3.session.Session', self.boto3_session_mock):
- operator = EmrAddStepsOperator(
- task_id='test_task',
- job_flow_name='test_cluster',
- cluster_states=['RUNNING', 'WAITING'],
- aws_conn_id='aws_default',
- dag=DAG('test_dag_id', default_args=self.args)
- )
+ with patch('airflow.contrib.hooks.emr_hook.EmrHook.get_cluster_id_by_name') \
+ as mock_get_cluster_id_by_name:
+ mock_get_cluster_id_by_name.return_value = expected_job_flow_id
- operator.execute(self.mock_context)
+ operator = EmrAddStepsOperator(
+ task_id='test_task',
+ job_flow_name='test_cluster',
+ cluster_states=['RUNNING', 'WAITING'],
+ aws_conn_id='aws_default',
+ dag=DAG('test_dag_id', default_args=self.args)
+ )
- ti = self.mock_context['ti']
+ operator.execute(self.mock_context)
- ti.xcom_push.assert_any_call(key='job_flow_id', value=expected_job_flow_id)
+ ti = self.mock_context['ti']
+
+ ti.xcom_push.assert_any_call(key='job_flow_id', value=expected_job_flow_id)
Review comment:
Can we change this to `assert_called_once_with`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/d0fbb523d617316a35461e6ee53781a7a716092c?src=pr&el=desc) will **decrease** coverage by `0.28%`.
> The diff coverage is `100%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 84.34% -0.29%
==========================================
Files 679 679
Lines 38497 38498 +1
==========================================
- Hits 32578 32471 -107
- Misses 5919 6027 +108
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <100%> (+0.25%)` | :arrow_up: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.75% <0%> (-20%)` | :arrow_down: |
| [airflow/utils/log/file\_processor\_handler.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvZmlsZV9wcm9jZXNzb3JfaGFuZGxlci5weQ==) | `88% <0%> (+2.66%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [d0fbb52...7c316cb](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] nuclearpinguin commented on a change in pull request
#6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test &
add test case for missing cluster
Posted by GitBox <gi...@apache.org>.
nuclearpinguin commented on a change in pull request #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test case for missing cluster
URL: https://github.com/apache/airflow/pull/6898#discussion_r361807954
##########
File path: airflow/contrib/operators/emr_add_steps_operator.py
##########
@@ -68,12 +68,16 @@ def __init__(
self.steps = steps
def execute(self, context):
- emr = EmrHook(aws_conn_id=self.aws_conn_id).get_conn()
+ emr_hook = EmrHook(aws_conn_id=self.aws_conn_id)
+
+ emr = emr_hook.get_conn()
job_flow_id = self.job_flow_id
if not job_flow_id:
- job_flow_id = emr.get_cluster_id_by_name(self.job_flow_name, self.cluster_states)
+ job_flow_id = emr_hook.get_cluster_id_by_name(self.job_flow_name, self.cluster_states)
+ if not job_flow_id:
+ raise AirflowException('No cluster found for name = %s' % self.job_flow_name)
Review comment:
```suggestion
raise AirflowException(f'No cluster found for name: {self.job_flow_name}')
```
But here we can use f-strings ✅
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] aviemzur commented on issue #6898: [AIRFLOW-6432] fix
EmrAddStepsOperator broken ref & faulty test & add test case for missing
cluster
Posted by GitBox <gi...@apache.org>.
aviemzur commented on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test case for missing cluster
URL: https://github.com/apache/airflow/pull/6898#issuecomment-569907727
> Waht do you mean by "fix EmrAddStepsOperator broken ref"? I can't quite decipher this.
>
> Can you please update the PR title to be a line you would like to see in the changelog about this.
Sure, how about:
```[AIRFLOW-6432] fix broken reference error in EmrAddStepsOperator when using job_flow_name and no cluster is found by that name```
or is this too wordy?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] nuclearpinguin merged pull request #6898: [AIRFLOW-6432]
Raise appropriate exception in EmrAddStepsOperator when using job_flow_name
and no cluster is found
Posted by GitBox <gi...@apache.org>.
nuclearpinguin merged pull request #6898: [AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator when using job_flow_name and no cluster is found
URL: https://github.com/apache/airflow/pull/6898
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/53aa9753c14a276ba372231f3d965df84731456e?src=pr&el=desc) will **decrease** coverage by `0.77%`.
> The diff coverage is `100%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.53% 83.75% -0.78%
==========================================
Files 672 679 +7
Lines 38153 38498 +345
==========================================
- Hits 32252 32244 -8
- Misses 5901 6254 +353
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <100%> (+0.25%)` | :arrow_up: |
| [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/contrib/hooks/jira\_hook.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2ppcmFfaG9vay5weQ==) | `0% <0%> (-80%)` | :arrow_down: |
| [airflow/contrib/operators/jira\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9qaXJhX29wZXJhdG9yLnB5) | `0% <0%> (-76.67%)` | :arrow_down: |
| [airflow/contrib/sensors/jira\_sensor.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL3NlbnNvcnMvamlyYV9zZW5zb3IucHk=) | `0% <0%> (-60%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| ... and [83 more](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [53aa975...2920ebd](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test
case for missing cluster
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test case for missing cluster
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/53aa9753c14a276ba372231f3d965df84731456e?src=pr&el=desc) will **decrease** coverage by `0.19%`.
> The diff coverage is `87.5%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
=========================================
- Coverage 84.53% 84.33% -0.2%
=========================================
Files 672 679 +7
Lines 38153 38509 +356
=========================================
+ Hits 32252 32477 +225
- Misses 5901 6032 +131
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.89% <100%> (+0.71%)` | :arrow_up: |
| [airflow/contrib/hooks/emr\_hook.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2Vtcl9ob29rLnB5) | `93.75% <66.66%> (+0.2%)` | :arrow_up: |
| [airflow/contrib/hooks/jira\_hook.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2ppcmFfaG9vay5weQ==) | `0% <0%> (-80%)` | :arrow_down: |
| [airflow/contrib/operators/jira\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9qaXJhX29wZXJhdG9yLnB5) | `0% <0%> (-76.67%)` | :arrow_down: |
| [airflow/contrib/sensors/jira\_sensor.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL3NlbnNvcnMvamlyYV9zZW5zb3IucHk=) | `0% <0%> (-60%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.75% <0%> (-19.97%)` | :arrow_down: |
| ... and [77 more](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [53aa975...d5d7fc0](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/d0fbb523d617316a35461e6ee53781a7a716092c?src=pr&el=desc) will **decrease** coverage by `0.86%`.
> The diff coverage is `100%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 83.75% -0.87%
==========================================
Files 679 679
Lines 38497 38498 +1
==========================================
- Hits 32578 32244 -334
- Misses 5919 6254 +335
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <100%> (+0.25%)` | :arrow_up: |
| [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `49.66% <0%> (-38.93%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| ... and [10 more](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [d0fbb52...2920ebd](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/d0fbb523d617316a35461e6ee53781a7a716092c?src=pr&el=desc) will **decrease** coverage by `0.86%`.
> The diff coverage is `100%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 83.75% -0.87%
==========================================
Files 679 679
Lines 38497 38498 +1
==========================================
- Hits 32578 32244 -334
- Misses 5919 6254 +335
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <100%> (+0.25%)` | :arrow_up: |
| [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `49.66% <0%> (-38.93%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| ... and [10 more](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [d0fbb52...2920ebd](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/d0fbb523d617316a35461e6ee53781a7a716092c?src=pr&el=desc) will **decrease** coverage by `0.86%`.
> The diff coverage is `100%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 83.75% -0.87%
==========================================
Files 679 679
Lines 38497 38498 +1
==========================================
- Hits 32578 32244 -334
- Misses 5919 6254 +335
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <100%> (+0.25%)` | :arrow_up: |
| [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `49.66% <0%> (-38.93%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| ... and [10 more](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [d0fbb52...2920ebd](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] aviemzur edited a comment on issue #6898: [AIRFLOW-6432]
raise appropriate exception in EmrAddStepsOperator when using job_flow_name
and no cluster is found by that name
Posted by GitBox <gi...@apache.org>.
aviemzur edited a comment on issue #6898: [AIRFLOW-6432] raise appropriate exception in EmrAddStepsOperator when using job_flow_name and no cluster is found by that name
URL: https://github.com/apache/airflow/pull/6898#issuecomment-569907727
> Waht do you mean by "fix EmrAddStepsOperator broken ref"? I can't quite decipher this.
>
> Can you please update the PR title to be a line you would like to see in the changelog about this.
Sure, how about:
```[AIRFLOW-6432] raise appropriate exception in EmrAddStepsOperator when using job_flow_name and no cluster is found by that name```
or is this too wordy?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] nuclearpinguin commented on a change in pull request
#6898: [AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator
when using job_flow_name and no cluster is found by that name
Posted by GitBox <gi...@apache.org>.
nuclearpinguin commented on a change in pull request #6898: [AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator when using job_flow_name and no cluster is found by that name
URL: https://github.com/apache/airflow/pull/6898#discussion_r362238354
##########
File path: airflow/contrib/operators/emr_add_steps_operator.py
##########
@@ -68,12 +68,14 @@ def __init__(
self.steps = steps
def execute(self, context):
- emr = EmrHook(aws_conn_id=self.aws_conn_id).get_conn()
+ emr_hook = EmrHook(aws_conn_id=self.aws_conn_id)
- job_flow_id = self.job_flow_id
+ emr = emr_hook.get_conn()
+ job_flow_id = self.job_flow_id or emr_hook.get_cluster_id_by_name(self.job_flow_name,
+ self.cluster_states)
if not job_flow_id:
- job_flow_id = emr.get_cluster_id_by_name(self.job_flow_name, self.cluster_states)
+ raise AirflowException(f'No cluster found for name: {self.job_flow_name}')
Review comment:
Now I start to think when this exception will be reached? When `emr_hook.get_cluster_id_by_name` return `None`. Is it possible or this will fail when calling `emr_hook.get_cluster_id_by_name` whith `self.job_flow_name=None`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] aviemzur edited a comment on issue #6898: [AIRFLOW-6432]
fix EmrAddStepsOperator broken ref & faulty test & add test case for
missing cluster
Posted by GitBox <gi...@apache.org>.
aviemzur edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test case for missing cluster
URL: https://github.com/apache/airflow/pull/6898#issuecomment-569907727
> Waht do you mean by "fix EmrAddStepsOperator broken ref"? I can't quite decipher this.
>
> Can you please update the PR title to be a line you would like to see in the changelog about this.
Sure, how about:
```[AIRFLOW-6432] throw proper exception in EmrAddStepsOperator when using job_flow_name and no cluster is found by that name```
or is this too wordy?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/d0fbb523d617316a35461e6ee53781a7a716092c?src=pr&el=desc) will **decrease** coverage by `0.86%`.
> The diff coverage is `100%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 83.75% -0.87%
==========================================
Files 679 679
Lines 38497 38498 +1
==========================================
- Hits 32578 32244 -334
- Misses 5919 6254 +335
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <100%> (+0.25%)` | :arrow_up: |
| [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `49.66% <0%> (-38.93%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| ... and [10 more](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [d0fbb52...2920ebd](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator when
using job_flow_name and no cluster is found
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator when using job_flow_name and no cluster is found
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/581a39575108e4cfe936f10644c62b1911087948?src=pr&el=desc) will **decrease** coverage by `0.07%`.
> The diff coverage is `85.71%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 84.54% -0.08%
==========================================
Files 679 679
Lines 38505 38618 +113
==========================================
+ Hits 32584 32650 +66
- Misses 5921 5968 +47
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <100%> (+0.25%)` | :arrow_up: |
| [airflow/contrib/hooks/emr\_hook.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2Vtcl9ob29rLnB5) | `93.75% <66.66%> (+0.2%)` | :arrow_up: |
| [airflow/contrib/hooks/cassandra\_hook.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2Nhc3NhbmRyYV9ob29rLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.75% <0%> (-20%)` | :arrow_down: |
| [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `91.3% <0%> (-8.7%)` | :arrow_down: |
| [airflow/utils/cli.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9jbGkucHk=) | `68.75% <0%> (-5.99%)` | :arrow_down: |
| ... and [44 more](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [581a395...d1d638f](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #6898: [AIRFLOW-6432] fix
EmrAddStepsOperator broken ref & faulty test
Posted by GitBox <gi...@apache.org>.
codecov-io commented on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/d0fbb523d617316a35461e6ee53781a7a716092c?src=pr&el=desc) will **decrease** coverage by `0.86%`.
> The diff coverage is `100%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 83.75% -0.87%
==========================================
Files 679 679
Lines 38497 38498 +1
==========================================
- Hits 32578 32244 -334
- Misses 5919 6254 +335
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <100%> (+0.25%)` | :arrow_up: |
| [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `49.66% <0%> (-38.93%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| ... and [10 more](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [d0fbb52...2920ebd](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/d0fbb523d617316a35461e6ee53781a7a716092c?src=pr&el=desc) will **decrease** coverage by `0.28%`.
> The diff coverage is `100%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 84.34% -0.29%
==========================================
Files 679 679
Lines 38497 38498 +1
==========================================
- Hits 32578 32471 -107
- Misses 5919 6027 +108
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <100%> (+0.25%)` | :arrow_up: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.75% <0%> (-20%)` | :arrow_down: |
| [airflow/utils/log/file\_processor\_handler.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvZmlsZV9wcm9jZXNzb3JfaGFuZGxlci5weQ==) | `88% <0%> (+2.66%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [d0fbb52...7c316cb](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/d0fbb523d617316a35461e6ee53781a7a716092c?src=pr&el=desc) will **decrease** coverage by `0.28%`.
> The diff coverage is `100%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 84.34% -0.29%
==========================================
Files 679 679
Lines 38497 38498 +1
==========================================
- Hits 32578 32471 -107
- Misses 5919 6027 +108
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <100%> (+0.25%)` | :arrow_up: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.75% <0%> (-20%)` | :arrow_down: |
| [airflow/utils/log/file\_processor\_handler.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvZmlsZV9wcm9jZXNzb3JfaGFuZGxlci5weQ==) | `88% <0%> (+2.66%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [d0fbb52...7c316cb](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] aviemzur closed pull request #6898: [AIRFLOW-6432] Raise
appropriate exception in EmrAddStepsOperator when using job_flow_name and
no cluster is found by that name
Posted by GitBox <gi...@apache.org>.
aviemzur closed pull request #6898: [AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator when using job_flow_name and no cluster is found by that name
URL: https://github.com/apache/airflow/pull/6898
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] aviemzur opened a new pull request #6898: [AIRFLOW-6432]
Raise appropriate exception in EmrAddStepsOperator when using job_flow_name
and no cluster is found by that name
Posted by GitBox <gi...@apache.org>.
aviemzur opened a new pull request #6898: [AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator when using job_flow_name and no cluster is found by that name
URL: https://github.com/apache/airflow/pull/6898
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references them in the PR title. For example, "\[AIRFLOW-6432\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-6432
- In case you are fixing a typo in the documentation you can prepend your commit with \[AIRFLOW-6432\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [x] Here are some details about my PR, including screenshots of any UI changes:
This PR fiixes EmrAddStepsOperator broken ref & faulty test
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for this extremely good reason:
Fix in `tests.contrib.operators.test_emr_add_steps_operator.TestEmrAddStepsOperator#test_init_with_cluster_name`
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes how to use it.
- All the public functions and the classes in the PR contain docstrings that explain what it does
- If you implement backwards incompatible changes, please leave a note in the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so we can assign it to a appropriate release
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/d0fbb523d617316a35461e6ee53781a7a716092c?src=pr&el=desc) will **decrease** coverage by `0.86%`.
> The diff coverage is `100%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 83.75% -0.87%
==========================================
Files 679 679
Lines 38497 38498 +1
==========================================
- Hits 32578 32244 -334
- Misses 5919 6254 +335
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <100%> (+0.25%)` | :arrow_up: |
| [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `49.66% <0%> (-38.93%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| ... and [10 more](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [d0fbb52...2920ebd](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] aviemzur commented on issue #6898: [AIRFLOW-6432] fix
EmrAddStepsOperator broken ref & faulty test
Posted by GitBox <gi...@apache.org>.
aviemzur commented on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568875294
@nuclearpinguin added commit [changes after CR #1](https://github.com/apache/airflow/pull/6898/commits/7c316cb7a5323e640df5402c43a4d7098677fe1c) with the changes you requested
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test
case for missing cluster
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test case for missing cluster
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/581a39575108e4cfe936f10644c62b1911087948?src=pr&el=desc) will **decrease** coverage by `0.28%`.
> The diff coverage is `87.5%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 84.33% -0.29%
==========================================
Files 679 679
Lines 38505 38509 +4
==========================================
- Hits 32584 32477 -107
- Misses 5921 6032 +111
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.89% <100%> (+0.71%)` | :arrow_up: |
| [airflow/contrib/hooks/emr\_hook.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2Vtcl9ob29rLnB5) | `93.75% <66.66%> (+0.2%)` | :arrow_up: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.75% <0%> (-20%)` | :arrow_down: |
| [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `91.59% <0%> (-0.29%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [581a395...d5d7fc0](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] aviemzur edited a comment on issue #6898: [AIRFLOW-6432]
throw proper exception in EmrAddStepsOperator when using job_flow_name and
no cluster is found by that name
Posted by GitBox <gi...@apache.org>.
aviemzur edited a comment on issue #6898: [AIRFLOW-6432] throw proper exception in EmrAddStepsOperator when using job_flow_name and no cluster is found by that name
URL: https://github.com/apache/airflow/pull/6898#issuecomment-569907727
> Waht do you mean by "fix EmrAddStepsOperator broken ref"? I can't quite decipher this.
>
> Can you please update the PR title to be a line you would like to see in the changelog about this.
Sure, how about:
```[AIRFLOW-6432] raise proper exception in EmrAddStepsOperator when using job_flow_name and no cluster is found by that name```
or is this too wordy?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test
case for missing cluster
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test case for missing cluster
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/581a39575108e4cfe936f10644c62b1911087948?src=pr&el=desc) will **decrease** coverage by `0.28%`.
> The diff coverage is `87.5%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 84.33% -0.29%
==========================================
Files 679 679
Lines 38505 38509 +4
==========================================
- Hits 32584 32477 -107
- Misses 5921 6032 +111
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.89% <100%> (+0.71%)` | :arrow_up: |
| [airflow/contrib/hooks/emr\_hook.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2Vtcl9ob29rLnB5) | `93.75% <66.66%> (+0.2%)` | :arrow_up: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.75% <0%> (-20%)` | :arrow_down: |
| [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `91.59% <0%> (-0.29%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [581a395...d5d7fc0](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] aviemzur commented on a change in pull request #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
Posted by GitBox <gi...@apache.org>.
aviemzur commented on a change in pull request #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898#discussion_r361288982
##########
File path: tests/contrib/operators/test_emr_add_steps_operator.py
##########
@@ -107,23 +107,26 @@ def test_execute_returns_step_id(self):
def test_init_with_cluster_name(self):
expected_job_flow_id = 'j-1231231234'
- self.emr_client_mock.get_cluster_id_by_name.return_value = expected_job_flow_id
self.emr_client_mock.add_job_flow_steps.return_value = ADD_STEPS_SUCCESS_RETURN
with patch('boto3.session.Session', self.boto3_session_mock):
- operator = EmrAddStepsOperator(
- task_id='test_task',
- job_flow_name='test_cluster',
- cluster_states=['RUNNING', 'WAITING'],
- aws_conn_id='aws_default',
- dag=DAG('test_dag_id', default_args=self.args)
- )
+ with patch('airflow.contrib.hooks.emr_hook.EmrHook.get_cluster_id_by_name') \
+ as mock_get_cluster_id_by_name:
+ mock_get_cluster_id_by_name.return_value = expected_job_flow_id
- operator.execute(self.mock_context)
+ operator = EmrAddStepsOperator(
+ task_id='test_task',
+ job_flow_name='test_cluster',
+ cluster_states=['RUNNING', 'WAITING'],
+ aws_conn_id='aws_default',
+ dag=DAG('test_dag_id', default_args=self.args)
+ )
- ti = self.mock_context['ti']
+ operator.execute(self.mock_context)
- ti.xcom_push.assert_any_call(key='job_flow_id', value=expected_job_flow_id)
+ ti = self.mock_context['ti']
+
+ ti.xcom_push.assert_any_call(key='job_flow_id', value=expected_job_flow_id)
Review comment:
Sure
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] aviemzur commented on a change in pull request #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test
case for missing cluster
Posted by GitBox <gi...@apache.org>.
aviemzur commented on a change in pull request #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test case for missing cluster
URL: https://github.com/apache/airflow/pull/6898#discussion_r362226904
##########
File path: airflow/contrib/operators/emr_add_steps_operator.py
##########
@@ -68,12 +68,16 @@ def __init__(
self.steps = steps
def execute(self, context):
- emr = EmrHook(aws_conn_id=self.aws_conn_id).get_conn()
+ emr_hook = EmrHook(aws_conn_id=self.aws_conn_id)
+
+ emr = emr_hook.get_conn()
job_flow_id = self.job_flow_id
if not job_flow_id:
- job_flow_id = emr.get_cluster_id_by_name(self.job_flow_name, self.cluster_states)
+ job_flow_id = emr_hook.get_cluster_id_by_name(self.job_flow_name, self.cluster_states)
+ if not job_flow_id:
Review comment:
Sure
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] nuclearpinguin commented on a change in pull request
#6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test &
add test case for missing cluster
Posted by GitBox <gi...@apache.org>.
nuclearpinguin commented on a change in pull request #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test case for missing cluster
URL: https://github.com/apache/airflow/pull/6898#discussion_r361807897
##########
File path: airflow/contrib/hooks/emr_hook.py
##########
@@ -51,11 +51,12 @@ def get_cluster_id_by_name(self, emr_cluster_name, cluster_states):
if len(matching_clusters) == 1:
cluster_id = matching_clusters[0]['Id']
- self.log.info('Found cluster name = %s id = %s' % (emr_cluster_name, cluster_id))
+ self.log.info(f'Found cluster name = {emr_cluster_name} id = {cluster_id}')
return cluster_id
elif len(matching_clusters) > 1:
- raise AirflowException('More than one cluster found for name = %s' % emr_cluster_name)
+ raise AirflowException(f'More than one cluster found for name {emr_cluster_name}')
Review comment:
```suggestion
raise AirflowException('More than one cluster found for name %s', emr_cluster_name)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test
case for missing cluster
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test case for missing cluster
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/581a39575108e4cfe936f10644c62b1911087948?src=pr&el=desc) will **decrease** coverage by `0.28%`.
> The diff coverage is `87.5%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 84.33% -0.29%
==========================================
Files 679 679
Lines 38505 38509 +4
==========================================
- Hits 32584 32477 -107
- Misses 5921 6032 +111
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.89% <100%> (+0.71%)` | :arrow_up: |
| [airflow/contrib/hooks/emr\_hook.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2Vtcl9ob29rLnB5) | `93.75% <66.66%> (+0.2%)` | :arrow_up: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.75% <0%> (-20%)` | :arrow_down: |
| [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `91.59% <0%> (-0.29%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [581a395...d5d7fc0](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator when
using job_flow_name and no cluster is found
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator when using job_flow_name and no cluster is found
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/581a39575108e4cfe936f10644c62b1911087948?src=pr&el=desc) will **decrease** coverage by `0.07%`.
> The diff coverage is `85.71%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 84.54% -0.08%
==========================================
Files 679 679
Lines 38505 38618 +113
==========================================
+ Hits 32584 32650 +66
- Misses 5921 5968 +47
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <100%> (+0.25%)` | :arrow_up: |
| [airflow/contrib/hooks/emr\_hook.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2Vtcl9ob29rLnB5) | `93.75% <66.66%> (+0.2%)` | :arrow_up: |
| [airflow/contrib/hooks/cassandra\_hook.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2Nhc3NhbmRyYV9ob29rLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.75% <0%> (-20%)` | :arrow_down: |
| [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `91.3% <0%> (-8.7%)` | :arrow_down: |
| [airflow/utils/cli.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9jbGkucHk=) | `68.75% <0%> (-5.99%)` | :arrow_down: |
| ... and [44 more](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [581a395...d1d638f](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/d0fbb523d617316a35461e6ee53781a7a716092c?src=pr&el=desc) will **decrease** coverage by `0.28%`.
> The diff coverage is `100%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 84.33% -0.29%
==========================================
Files 679 679
Lines 38497 38498 +1
==========================================
- Hits 32578 32469 -109
- Misses 5919 6029 +110
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <100%> (+0.25%)` | :arrow_up: |
| [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `100% <0%> (ø)` | :arrow_up: |
| [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `100% <0%> (ø)` | :arrow_up: |
| [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `100% <0%> (ø)` | :arrow_up: |
| [airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==) | `100% <0%> (ø)` | :arrow_up: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `88.59% <0%> (ø)` | :arrow_up: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| ... and [10 more](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [d0fbb52...2920ebd](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator when
using job_flow_name and no cluster is found
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator when using job_flow_name and no cluster is found
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/53aa9753c14a276ba372231f3d965df84731456e?src=pr&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `85.71%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
+ Coverage 84.53% 84.54% +0.01%
==========================================
Files 672 679 +7
Lines 38153 38618 +465
==========================================
+ Hits 32252 32650 +398
- Misses 5901 5968 +67
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <100%> (+0.25%)` | :arrow_up: |
| [airflow/contrib/hooks/emr\_hook.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2Vtcl9ob29rLnB5) | `93.75% <66.66%> (+0.2%)` | :arrow_up: |
| [airflow/contrib/hooks/cassandra\_hook.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2Nhc3NhbmRyYV9ob29rLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/contrib/hooks/jira\_hook.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2ppcmFfaG9vay5weQ==) | `0% <0%> (-80%)` | :arrow_down: |
| [airflow/contrib/operators/jira\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9qaXJhX29wZXJhdG9yLnB5) | `0% <0%> (-76.67%)` | :arrow_down: |
| [airflow/contrib/sensors/jira\_sensor.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL3NlbnNvcnMvamlyYV9zZW5zb3IucHk=) | `0% <0%> (-60%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| ... and [112 more](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [53aa975...d1d638f](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] nuclearpinguin commented on a change in pull request
#6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test &
add test case for missing cluster
Posted by GitBox <gi...@apache.org>.
nuclearpinguin commented on a change in pull request #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test case for missing cluster
URL: https://github.com/apache/airflow/pull/6898#discussion_r361807885
##########
File path: airflow/contrib/hooks/emr_hook.py
##########
@@ -51,11 +51,12 @@ def get_cluster_id_by_name(self, emr_cluster_name, cluster_states):
if len(matching_clusters) == 1:
cluster_id = matching_clusters[0]['Id']
- self.log.info('Found cluster name = %s id = %s' % (emr_cluster_name, cluster_id))
+ self.log.info(f'Found cluster name = {emr_cluster_name} id = {cluster_id}')
Review comment:
```suggestion
self.log.info('Found cluster name = %s id = %s', emr_cluster_name, cluster_id)
```
In case of logging we do formatting this way.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/d0fbb523d617316a35461e6ee53781a7a716092c?src=pr&el=desc) will **decrease** coverage by `0.86%`.
> The diff coverage is `100%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 83.75% -0.87%
==========================================
Files 679 679
Lines 38497 38498 +1
==========================================
- Hits 32578 32244 -334
- Misses 5919 6254 +335
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <100%> (+0.25%)` | :arrow_up: |
| [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `49.66% <0%> (-38.93%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| ... and [10 more](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [d0fbb52...2920ebd](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ashb commented on issue #6898: [AIRFLOW-6432] fix
EmrAddStepsOperator broken ref & faulty test & add test case for missing
cluster
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test case for missing cluster
URL: https://github.com/apache/airflow/pull/6898#issuecomment-569898226
Waht do you mean by "fix EmrAddStepsOperator broken ref"? I can't quite decipher this.
Can you please update the PR title to be a line you would like to see in the changelog about this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] nuclearpinguin commented on a change in pull request
#6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test &
add test case for missing cluster
Posted by GitBox <gi...@apache.org>.
nuclearpinguin commented on a change in pull request #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test case for missing cluster
URL: https://github.com/apache/airflow/pull/6898#discussion_r362162056
##########
File path: tests/contrib/operators/test_emr_add_steps_operator.py
##########
@@ -107,23 +108,45 @@ def test_execute_returns_step_id(self):
def test_init_with_cluster_name(self):
expected_job_flow_id = 'j-1231231234'
- self.emr_client_mock.get_cluster_id_by_name.return_value = expected_job_flow_id
self.emr_client_mock.add_job_flow_steps.return_value = ADD_STEPS_SUCCESS_RETURN
with patch('boto3.session.Session', self.boto3_session_mock):
+ with patch('airflow.contrib.hooks.emr_hook.EmrHook.get_cluster_id_by_name') \
+ as mock_get_cluster_id_by_name:
+ mock_get_cluster_id_by_name.return_value = expected_job_flow_id
+
+ operator = EmrAddStepsOperator(
+ task_id='test_task',
+ job_flow_name='test_cluster',
+ cluster_states=['RUNNING', 'WAITING'],
+ aws_conn_id='aws_default',
+ dag=DAG('test_dag_id', default_args=self.args)
+ )
+
+ operator.execute(self.mock_context)
+
+ ti = self.mock_context['ti']
+
+ ti.xcom_push.assert_called_once_with(key='job_flow_id', value=expected_job_flow_id)
+
+ def test_init_with_nonexistent_cluster_name(self):
+ cluster_name = 'test_cluster'
+
+ with patch('airflow.contrib.hooks.emr_hook.EmrHook.get_cluster_id_by_name') \
+ as mock_get_cluster_id_by_name:
+ mock_get_cluster_id_by_name.return_value = None
+
operator = EmrAddStepsOperator(
task_id='test_task',
- job_flow_name='test_cluster',
+ job_flow_name=cluster_name,
cluster_states=['RUNNING', 'WAITING'],
aws_conn_id='aws_default',
dag=DAG('test_dag_id', default_args=self.args)
)
- operator.execute(self.mock_context)
-
- ti = self.mock_context['ti']
-
- ti.xcom_push.assert_any_call(key='job_flow_id', value=expected_job_flow_id)
+ with self.assertRaises(AirflowException) as error:
+ operator.execute(self.mock_context)
+ self.assertEqual(str(error.exception), 'No cluster found for name = test_cluster')
Review comment:
```suggestion
self.assertEqual(str(error.exception), f'No cluster found for name: {cluster_name}')
```
The test need an adjustment after the change in core :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] nuclearpinguin commented on a change in pull request
#6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test &
add test case for missing cluster
Posted by GitBox <gi...@apache.org>.
nuclearpinguin commented on a change in pull request #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test case for missing cluster
URL: https://github.com/apache/airflow/pull/6898#discussion_r361808039
##########
File path: airflow/contrib/operators/emr_add_steps_operator.py
##########
@@ -68,12 +68,16 @@ def __init__(
self.steps = steps
def execute(self, context):
- emr = EmrHook(aws_conn_id=self.aws_conn_id).get_conn()
+ emr_hook = EmrHook(aws_conn_id=self.aws_conn_id)
+
+ emr = emr_hook.get_conn()
job_flow_id = self.job_flow_id
if not job_flow_id:
- job_flow_id = emr.get_cluster_id_by_name(self.job_flow_name, self.cluster_states)
+ job_flow_id = emr_hook.get_cluster_id_by_name(self.job_flow_name, self.cluster_states)
+ if not job_flow_id:
Review comment:
There is duplicated check, what about:
```python
job_flow_id = job_flow_id or emr_hook.get_cluster_id_by_name(self.job_flow_name, self.cluster_states)
if not job_flow_id:
raise...
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] aviemzur commented on a change in pull request #6898:
[AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator when
using job_flow_name and no cluster is found by that name
Posted by GitBox <gi...@apache.org>.
aviemzur commented on a change in pull request #6898: [AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator when using job_flow_name and no cluster is found by that name
URL: https://github.com/apache/airflow/pull/6898#discussion_r362238595
##########
File path: airflow/contrib/operators/emr_add_steps_operator.py
##########
@@ -68,12 +68,14 @@ def __init__(
self.steps = steps
def execute(self, context):
- emr = EmrHook(aws_conn_id=self.aws_conn_id).get_conn()
+ emr_hook = EmrHook(aws_conn_id=self.aws_conn_id)
- job_flow_id = self.job_flow_id
+ emr = emr_hook.get_conn()
+ job_flow_id = self.job_flow_id or emr_hook.get_cluster_id_by_name(self.job_flow_name,
+ self.cluster_states)
if not job_flow_id:
- job_flow_id = emr.get_cluster_id_by_name(self.job_flow_name, self.cluster_states)
+ raise AirflowException(f'No cluster found for name: {self.job_flow_name}')
Review comment:
If you ask for a cluster that doesn't exist it returns `None`
https://github.com/apache/airflow/blob/master/airflow/contrib/hooks/emr_hook.py#L52-L59
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/d0fbb523d617316a35461e6ee53781a7a716092c?src=pr&el=desc) will **decrease** coverage by `0.86%`.
> The diff coverage is `100%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 83.75% -0.87%
==========================================
Files 679 679
Lines 38497 38498 +1
==========================================
- Hits 32578 32244 -334
- Misses 5919 6254 +335
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <100%> (+0.25%)` | :arrow_up: |
| [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `49.66% <0%> (-38.93%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| ... and [10 more](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [d0fbb52...2920ebd](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] aviemzur edited a comment on issue #6898: [AIRFLOW-6432]
Raise appropriate exception in EmrAddStepsOperator when using job_flow_name
and no cluster is found by that name
Posted by GitBox <gi...@apache.org>.
aviemzur edited a comment on issue #6898: [AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator when using job_flow_name and no cluster is found by that name
URL: https://github.com/apache/airflow/pull/6898#issuecomment-569907727
> Waht do you mean by "fix EmrAddStepsOperator broken ref"? I can't quite decipher this.
>
> Can you please update the PR title to be a line you would like to see in the changelog about this.
Sure, how about:
```[AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator when using job_flow_name and no cluster is found by that name```
or is this too wordy?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test
case for missing cluster
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test case for missing cluster
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/581a39575108e4cfe936f10644c62b1911087948?src=pr&el=desc) will **decrease** coverage by `0.28%`.
> The diff coverage is `87.5%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 84.33% -0.29%
==========================================
Files 679 679
Lines 38505 38509 +4
==========================================
- Hits 32584 32477 -107
- Misses 5921 6032 +111
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.89% <100%> (+0.71%)` | :arrow_up: |
| [airflow/contrib/hooks/emr\_hook.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2Vtcl9ob29rLnB5) | `93.75% <66.66%> (+0.2%)` | :arrow_up: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.75% <0%> (-20%)` | :arrow_down: |
| [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `91.59% <0%> (-0.29%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [581a395...d5d7fc0](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/d0fbb523d617316a35461e6ee53781a7a716092c?src=pr&el=desc) will **decrease** coverage by `0.86%`.
> The diff coverage is `100%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 83.75% -0.87%
==========================================
Files 679 679
Lines 38497 38498 +1
==========================================
- Hits 32578 32244 -334
- Misses 5919 6254 +335
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <100%> (+0.25%)` | :arrow_up: |
| [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `49.66% <0%> (-38.93%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| ... and [10 more](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [d0fbb52...2920ebd](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] nuclearpinguin commented on a change in pull request
#6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test &
add test case for missing cluster
Posted by GitBox <gi...@apache.org>.
nuclearpinguin commented on a change in pull request #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test & add test case for missing cluster
URL: https://github.com/apache/airflow/pull/6898#discussion_r361807913
##########
File path: airflow/contrib/hooks/emr_hook.py
##########
@@ -51,11 +51,12 @@ def get_cluster_id_by_name(self, emr_cluster_name, cluster_states):
if len(matching_clusters) == 1:
cluster_id = matching_clusters[0]['Id']
- self.log.info('Found cluster name = %s id = %s' % (emr_cluster_name, cluster_id))
+ self.log.info(f'Found cluster name = {emr_cluster_name} id = {cluster_id}')
return cluster_id
elif len(matching_clusters) > 1:
- raise AirflowException('More than one cluster found for name = %s' % emr_cluster_name)
+ raise AirflowException(f'More than one cluster found for name {emr_cluster_name}')
else:
+ self.log.info(f'No cluster found for name {emr_cluster_name}')
Review comment:
```suggestion
self.log.info('No cluster found for name %s', emr_cluster_name)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] aviemzur commented on a change in pull request #6898:
[AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator when
using job_flow_name and no cluster is found by that name
Posted by GitBox <gi...@apache.org>.
aviemzur commented on a change in pull request #6898: [AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator when using job_flow_name and no cluster is found by that name
URL: https://github.com/apache/airflow/pull/6898#discussion_r362238595
##########
File path: airflow/contrib/operators/emr_add_steps_operator.py
##########
@@ -68,12 +68,14 @@ def __init__(
self.steps = steps
def execute(self, context):
- emr = EmrHook(aws_conn_id=self.aws_conn_id).get_conn()
+ emr_hook = EmrHook(aws_conn_id=self.aws_conn_id)
- job_flow_id = self.job_flow_id
+ emr = emr_hook.get_conn()
+ job_flow_id = self.job_flow_id or emr_hook.get_cluster_id_by_name(self.job_flow_name,
+ self.cluster_states)
if not job_flow_id:
- job_flow_id = emr.get_cluster_id_by_name(self.job_flow_name, self.cluster_states)
+ raise AirflowException(f'No cluster found for name: {self.job_flow_name}')
Review comment:
If you ask for a cluster that doesn't exist it returns `None`
[emr_hook.py](https://github.com/apache/airflow/blob/master/airflow/contrib/hooks/emr_hook.py#L52-L59)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] nuclearpinguin commented on a change in pull request
#6898: [AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator
when using job_flow_name and no cluster is found by that name
Posted by GitBox <gi...@apache.org>.
nuclearpinguin commented on a change in pull request #6898: [AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator when using job_flow_name and no cluster is found by that name
URL: https://github.com/apache/airflow/pull/6898#discussion_r362238354
##########
File path: airflow/contrib/operators/emr_add_steps_operator.py
##########
@@ -68,12 +68,14 @@ def __init__(
self.steps = steps
def execute(self, context):
- emr = EmrHook(aws_conn_id=self.aws_conn_id).get_conn()
+ emr_hook = EmrHook(aws_conn_id=self.aws_conn_id)
- job_flow_id = self.job_flow_id
+ emr = emr_hook.get_conn()
+ job_flow_id = self.job_flow_id or emr_hook.get_cluster_id_by_name(self.job_flow_name,
+ self.cluster_states)
if not job_flow_id:
- job_flow_id = emr.get_cluster_id_by_name(self.job_flow_name, self.cluster_states)
+ raise AirflowException(f'No cluster found for name: {self.job_flow_name}')
Review comment:
No I start to think when this exception will be reached? When `emr_hook.get_cluster_id_by_name` return `None`. Is it possible or this will fail when calling `emr_hook.get_cluster_id_by_name` whith `self.job_flow_name=None`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator when
using job_flow_name and no cluster is found
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] Raise appropriate exception in EmrAddStepsOperator when using job_flow_name and no cluster is found
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/53aa9753c14a276ba372231f3d965df84731456e?src=pr&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `85.71%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
+ Coverage 84.53% 84.54% +0.01%
==========================================
Files 672 679 +7
Lines 38153 38618 +465
==========================================
+ Hits 32252 32650 +398
- Misses 5901 5968 +67
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <100%> (+0.25%)` | :arrow_up: |
| [airflow/contrib/hooks/emr\_hook.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2Vtcl9ob29rLnB5) | `93.75% <66.66%> (+0.2%)` | :arrow_up: |
| [airflow/contrib/hooks/cassandra\_hook.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2Nhc3NhbmRyYV9ob29rLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/contrib/hooks/jira\_hook.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2ppcmFfaG9vay5weQ==) | `0% <0%> (-80%)` | :arrow_down: |
| [airflow/contrib/operators/jira\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9qaXJhX29wZXJhdG9yLnB5) | `0% <0%> (-76.67%)` | :arrow_down: |
| [airflow/contrib/sensors/jira\_sensor.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL3NlbnNvcnMvamlyYV9zZW5zb3IucHk=) | `0% <0%> (-60%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| ... and [112 more](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [53aa975...d1d638f](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] aviemzur commented on issue #6898: [AIRFLOW-6432] fix
EmrAddStepsOperator broken ref & faulty test
Posted by GitBox <gi...@apache.org>.
aviemzur commented on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568872345
R: @ashb
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6898:
[AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #6898: [AIRFLOW-6432] fix EmrAddStepsOperator broken ref & faulty test
URL: https://github.com/apache/airflow/pull/6898#issuecomment-568865752
# [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=h1) Report
> Merging [#6898](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/d0fbb523d617316a35461e6ee53781a7a716092c?src=pr&el=desc) will **decrease** coverage by `0.86%`.
> The diff coverage is `100%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6898/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #6898 +/- ##
==========================================
- Coverage 84.62% 83.75% -0.87%
==========================================
Files 679 679
Lines 38497 38498 +1
==========================================
- Hits 32578 32244 -334
- Misses 5919 6254 +335
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <100%> (+0.25%)` | :arrow_up: |
| [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `49.66% <0%> (-38.93%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| ... and [10 more](https://codecov.io/gh/apache/airflow/pull/6898/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=footer). Last update [d0fbb52...2920ebd](https://codecov.io/gh/apache/airflow/pull/6898?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services