You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/12/13 22:30:18 UTC
[GitHub] [airflow] subkanthi opened a new pull request #20271: Fix mypy airbyte provider errors
subkanthi opened a new pull request #20271:
URL: https://github.com/apache/airflow/pull/20271
Related: #19891
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #20271: Fix mypy airbyte provider errors
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #20271:
URL: https://github.com/apache/airflow/pull/20271
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #20271: Fix mypy airbyte provider errors
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #20271:
URL: https://github.com/apache/airflow/pull/20271#discussion_r768180838
##########
File path: airflow/providers/airbyte/hooks/airbyte.py
##########
@@ -71,7 +71,7 @@ def wait_for_job(
raise AirflowException(f"Timeout: Airbyte job {job_id} is not ready after {timeout}s")
time.sleep(wait_seconds)
try:
- job = self.get_job(job_id=job_id)
+ job = self.get_job(int(job_id=job_id))
Review comment:
I think you mean
```suggestion
job = self.get_job(job_id=int(job_id))
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #20271: Fix mypy airbyte provider errors
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #20271:
URL: https://github.com/apache/airflow/pull/20271#discussion_r768784219
##########
File path: airflow/providers/airbyte/sensors/airbyte.py
##########
@@ -54,7 +54,7 @@ def __init__(
def poke(self, context: dict) -> bool:
hook = AirbyteHook(airbyte_conn_id=self.airbyte_conn_id, api_version=self.api_version)
- job = hook.get_job(job_id=self.airbyte_job_id)
+ job = hook.get_job(job_id=int(self.airbyte_job_id))
Review comment:
done.
##########
File path: airflow/providers/airbyte/hooks/airbyte.py
##########
@@ -46,12 +46,12 @@ class AirbyteHook(HttpHook):
ERROR = "error"
INCOMPLETE = "incomplete"
- def __init__(self, airbyte_conn_id: str = "airbyte_default", api_version: Optional[str] = "v1") -> None:
+ def __init__(self, airbyte_conn_id: str = "airbyte_default", api_version: str = "v1") -> None:
super().__init__(http_conn_id=airbyte_conn_id)
self.api_version: str = api_version
def wait_for_job(
- self, job_id: str, wait_seconds: Optional[float] = 3, timeout: Optional[float] = 3600
+ self, job_id: str, wait_seconds: float = 3, timeout: Optional[float] = 3600
Review comment:
changed, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a change in pull request #20271: Fix mypy airbyte provider errors
Posted by GitBox <gi...@apache.org>.
josh-fell commented on a change in pull request #20271:
URL: https://github.com/apache/airflow/pull/20271#discussion_r768263731
##########
File path: airflow/providers/airbyte/sensors/airbyte.py
##########
@@ -54,7 +54,7 @@ def __init__(
def poke(self, context: dict) -> bool:
hook = AirbyteHook(airbyte_conn_id=self.airbyte_conn_id, api_version=self.api_version)
- job = hook.get_job(job_id=self.airbyte_job_id)
+ job = hook.get_job(job_id=int(self.airbyte_job_id))
Review comment:
Looks like `airbyte_job_id` in the sensor's constructor is still typed as a `str` rather than `int`. Changing the typing there should remove the need for the cast.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #20271: Fix mypy airbyte provider errors
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #20271:
URL: https://github.com/apache/airflow/pull/20271#discussion_r768207839
##########
File path: airflow/providers/airbyte/sensors/airbyte.py
##########
@@ -54,7 +54,7 @@ def __init__(
def poke(self, context: dict) -> bool:
hook = AirbyteHook(airbyte_conn_id=self.airbyte_conn_id, api_version=self.api_version)
- job = hook.get_job(job_id=self.airbyte_job_id)
+ job = hook.get_job(job_id=int(self.airbyte_job_id))
Review comment:
`def get_job(self, job_id: int) -> Any:`
This is fixing this error, sorry not sure why its a breaking change.
airflow/providers/airbyte/sensors/airbyte.py:57: error: Argument "job_id" to "get_job" of "AirbyteHook" has incompatible type "str"; expected "int"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #20271: Fix mypy airbyte provider errors
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #20271:
URL: https://github.com/apache/airflow/pull/20271#discussion_r768181401
##########
File path: airflow/providers/airbyte/hooks/airbyte.py
##########
@@ -46,12 +46,12 @@ class AirbyteHook(HttpHook):
ERROR = "error"
INCOMPLETE = "incomplete"
- def __init__(self, airbyte_conn_id: str = "airbyte_default", api_version: Optional[str] = "v1") -> None:
+ def __init__(self, airbyte_conn_id: str = "airbyte_default", api_version: str = "v1") -> None:
super().__init__(http_conn_id=airbyte_conn_id)
self.api_version: str = api_version
def wait_for_job(
- self, job_id: str, wait_seconds: Optional[float] = 3, timeout: Optional[float] = 3600
+ self, job_id: str, wait_seconds: float = 3, timeout: Optional[float] = 3600
Review comment:
We should probably do this too
```suggestion
self, job_id: Union[str, int], wait_seconds: float = 3, timeout: Optional[float] = 3600
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #20271: Fix mypy airbyte provider errors
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #20271:
URL: https://github.com/apache/airflow/pull/20271#discussion_r768181906
##########
File path: airflow/providers/airbyte/sensors/airbyte.py
##########
@@ -54,7 +54,7 @@ def __init__(
def poke(self, context: dict) -> bool:
hook = AirbyteHook(airbyte_conn_id=self.airbyte_conn_id, api_version=self.api_version)
- job = hook.get_job(job_id=self.airbyte_job_id)
+ job = hook.get_job(job_id=int(self.airbyte_job_id))
Review comment:
We shouldn't need to cast the job id to an int here -- if we do it's a breaking change.
```suggestion
job = hook.get_job(job_id=self.airbyte_job_id)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #20271: Fix mypy airbyte provider errors
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #20271:
URL: https://github.com/apache/airflow/pull/20271#discussion_r768208463
##########
File path: airflow/providers/airbyte/hooks/airbyte.py
##########
@@ -71,7 +71,7 @@ def wait_for_job(
raise AirflowException(f"Timeout: Airbyte job {job_id} is not ready after {timeout}s")
time.sleep(wait_seconds)
try:
- job = self.get_job(job_id=job_id)
+ job = self.get_job(int(job_id=job_id))
Review comment:
Yes, I was quick to mark the PR as draft because I noticed this :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org