You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/01/02 15:25:00 UTC
[GitHub] [airflow] kaxil opened a new pull request #7005: [AIRFLOW-6427] Fix
broken example_qubole_operator dag
kaxil opened a new pull request #7005: [AIRFLOW-6427] Fix broken example_qubole_operator dag
URL: https://github.com/apache/airflow/pull/7005
With the current example (https://github.com/apache/airflow/blob/aa90753cf5f64bf435044cf2e6b81a02fdcf6b33/airflow/contrib/example_dags/example_qubole_operator.py#L191), I get the following error:
```
airflow.exceptions.AirflowException: Invalid arguments were passed to QuboleOperator (task_id: db_import). Invalid arguments were:
*args: ()
**kwargs: {'db_parallelism': 2}
```
The `qds_sdk` contains the following:
```python
optparser.add_option("--parallelism", dest="db_parallelism",
help="Mode 1 and 2: Number of parallel threads to use for extracting data")
```
and hence it should be `parallelism` and not `db_parallelism`
---
- [x] Description above provides context of the change
- [x] Commit message contains [\[AIRFLOW-6427\]](https://issues.apache.org/jira/browse/AIRFLOW-6427)
- [x] Unit tests coverage for changes (not needed for documentation changes)
- [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
- [x] Relevant documentation is updated including usage instructions.
- [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7005:
[AIRFLOW-6427] Fix error in example_qubole_operator dag
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7005: [AIRFLOW-6427] Fix error in example_qubole_operator dag
URL: https://github.com/apache/airflow/pull/7005#issuecomment-570253096
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7005?src=pr&el=h1) Report
> Merging [#7005](https://codecov.io/gh/apache/airflow/pull/7005?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/aa90753cf5f64bf435044cf2e6b81a02fdcf6b33?src=pr&el=desc) will **decrease** coverage by `0.45%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/airflow/pull/7005?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7005 +/- ##
==========================================
- Coverage 84.85% 84.39% -0.46%
==========================================
Files 679 679
Lines 38542 38542
==========================================
- Hits 32703 32529 -174
- Misses 5839 6013 +174
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7005?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...ow/contrib/example\_dags/example\_qubole\_operator.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3F1Ym9sZV9vcGVyYXRvci5weQ==) | `90% <ø> (+10%)` | :arrow_up: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/sequential\_executor.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvc2VxdWVudGlhbF9leGVjdXRvci5weQ==) | `56% <0%> (-32%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.75% <0%> (-20%)` | :arrow_down: |
| [airflow/utils/helpers.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9oZWxwZXJzLnB5) | `74.5% <0%> (-7.19%)` | :arrow_down: |
| [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `80.95% <0%> (-7.05%)` | :arrow_down: |
| [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `90% <0%> (-6.67%)` | :arrow_down: |
| ... and [2 more](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7005?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7005?src=pr&el=footer). Last update [aa90753...805bd1e](https://codecov.io/gh/apache/airflow/pull/7005?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil merged pull request #7005: [AIRFLOW-6427] Fix error
in example_qubole_operator dag
Posted by GitBox <gi...@apache.org>.
kaxil merged pull request #7005: [AIRFLOW-6427] Fix error in example_qubole_operator dag
URL: https://github.com/apache/airflow/pull/7005
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #7005: [AIRFLOW-6427] Fix
error in example_qubole_operator dag
Posted by GitBox <gi...@apache.org>.
codecov-io commented on issue #7005: [AIRFLOW-6427] Fix error in example_qubole_operator dag
URL: https://github.com/apache/airflow/pull/7005#issuecomment-570253096
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7005?src=pr&el=h1) Report
> Merging [#7005](https://codecov.io/gh/apache/airflow/pull/7005?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/aa90753cf5f64bf435044cf2e6b81a02fdcf6b33?src=pr&el=desc) will **decrease** coverage by `0.45%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/airflow/pull/7005?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7005 +/- ##
==========================================
- Coverage 84.85% 84.39% -0.46%
==========================================
Files 679 679
Lines 38542 38542
==========================================
- Hits 32703 32529 -174
- Misses 5839 6013 +174
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7005?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...ow/contrib/example\_dags/example\_qubole\_operator.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3F1Ym9sZV9vcGVyYXRvci5weQ==) | `90% <ø> (+10%)` | :arrow_up: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/sequential\_executor.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvc2VxdWVudGlhbF9leGVjdXRvci5weQ==) | `56% <0%> (-32%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.75% <0%> (-20%)` | :arrow_down: |
| [airflow/utils/helpers.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9oZWxwZXJzLnB5) | `74.5% <0%> (-7.19%)` | :arrow_down: |
| [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `80.95% <0%> (-7.05%)` | :arrow_down: |
| [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `90% <0%> (-6.67%)` | :arrow_down: |
| ... and [2 more](https://codecov.io/gh/apache/airflow/pull/7005/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7005?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7005?src=pr&el=footer). Last update [aa90753...805bd1e](https://codecov.io/gh/apache/airflow/pull/7005?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services