You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "vincbeck (via GitHub)" <gi...@apache.org> on 2023/05/19 15:06:13 UTC
[GitHub] [airflow] vincbeck opened a new pull request, #31422: Add D400 pydocstyle check - Google provider only
vincbeck opened a new pull request, #31422:
URL: https://github.com/apache/airflow/pull/31422
This PR is a follow up of #31135. It had been decided to split #31135 in multiple PRs to make it easier to review. This PR scope is Provider Google only.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vincbeck commented on pull request #31422: Add D400 pydocstyle check - Google provider only
Posted by "vincbeck (via GitHub)" <gi...@apache.org>.
vincbeck commented on PR #31422:
URL: https://github.com/apache/airflow/pull/31422#issuecomment-1568634683
Thanks for the review @uranusjr . I just applied your suggestions. If you have time, I have many open PRs related to the same issue:
- #31423
- #31424
- #31425
- #31427
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a diff in pull request #31422: Add D400 pydocstyle check - Google provider only
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #31422:
URL: https://github.com/apache/airflow/pull/31422#discussion_r1201797951
##########
airflow/providers/google/cloud/transfers/trino_to_gcs.py:
##########
@@ -129,16 +129,15 @@ def fetchmany(self, size=None) -> list:
def __next__(self) -> Any:
"""
Return the next row from the currently executing SQL statement using the same semantics as
- ``.fetchone()``. A ``StopIteration`` exception is raised when the result set is exhausted.
- :return:
+ ``.fetchone()``. A ``StopIteration`` exception is raised when the result set is exhausted.
"""
Review Comment:
Should break line after `fetchone()` instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vincbeck commented on a diff in pull request #31422: Add D400 pydocstyle check - Google provider only
Posted by "vincbeck (via GitHub)" <gi...@apache.org>.
vincbeck commented on code in PR #31422:
URL: https://github.com/apache/airflow/pull/31422#discussion_r1202540059
##########
airflow/providers/google/cloud/transfers/presto_to_gcs.py:
##########
@@ -129,16 +129,15 @@ def fetchmany(self, size=None) -> list:
def __next__(self) -> Any:
"""
Return the next row from the currently executing SQL statement using the same semantics as
- ``.fetchone()``. A ``StopIteration`` exception is raised when the result set is exhausted.
- :return:
+ ``.fetchone()``. A ``StopIteration`` exception is raised when the result set is exhausted.
Review Comment:
There are not :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a diff in pull request #31422: Add D400 pydocstyle check - Google provider only
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #31422:
URL: https://github.com/apache/airflow/pull/31422#discussion_r1201800069
##########
airflow/providers/google/cloud/transfers/presto_to_gcs.py:
##########
@@ -129,16 +129,15 @@ def fetchmany(self, size=None) -> list:
def __next__(self) -> Any:
"""
Return the next row from the currently executing SQL statement using the same semantics as
- ``.fetchone()``. A ``StopIteration`` exception is raised when the result set is exhausted.
- :return:
+ ``.fetchone()``. A ``StopIteration`` exception is raised when the result set is exhausted.
Review Comment:
Same as above (not sure if there are more)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a diff in pull request #31422: Add D400 pydocstyle check - Google provider only
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #31422:
URL: https://github.com/apache/airflow/pull/31422#discussion_r1209784463
##########
airflow/providers/google/cloud/hooks/dataprep.py:
##########
@@ -59,6 +59,7 @@ class GoogleDataprepHook(BaseHook):
"""
Hook for connection with Dataprep API.
To get connection Dataprep with Airflow you need Dataprep token.
+
Review Comment:
```suggestion
To get connection Dataprep with Airflow you need Dataprep token.
```
##########
airflow/providers/google/cloud/sensors/gcs.py:
##########
@@ -132,7 +132,7 @@ class GCSObjectExistenceAsyncSensor(GCSObjectExistenceSensor):
"""
Checks for the existence of a file in Google Cloud Storage.
Review Comment:
```suggestion
Checks for the existence of a file in Google Cloud Storage.
```
##########
airflow/providers/google/cloud/hooks/os_login.py:
##########
@@ -14,9 +14,9 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-"""
-.. spelling::
+"""Spelling exceptions.
Review Comment:
```suggestion
"""OS Login hooks.
```
##########
airflow/providers/google/cloud/operators/dataflow.py:
##########
@@ -50,7 +50,7 @@ class CheckJobRunning(Enum):
Helper enum for choosing what to do if job is already running
Review Comment:
```suggestion
Helper enum for choosing what to do if job is already running.
```
##########
airflow/providers/google/cloud/operators/dataflow.py:
##########
@@ -50,7 +50,7 @@ class CheckJobRunning(Enum):
Helper enum for choosing what to do if job is already running
IgnoreJob - do not check if running
FinishIfRunning - finish current dag run with no action
- WaitForRun - wait for job to finish and then continue with new job
+ WaitForRun - wait for job to finish and then continue with new job.
Review Comment:
```suggestion
WaitForRun - wait for job to finish and then continue with new job
```
##########
airflow/providers/google/cloud/operators/dataprep.py:
##########
@@ -31,7 +31,8 @@
class DataprepGetJobsForJobGroupOperator(GoogleCloudBaseOperator):
"""
Get information about the batch jobs within a Cloud Dataprep job.
Review Comment:
```suggestion
Get information about the batch jobs within a Cloud Dataprep job.
```
##########
airflow/providers/google/cloud/operators/dataprep.py:
##########
@@ -125,8 +127,9 @@ class DataprepRunJobGroupOperator(GoogleCloudBaseOperator):
"""
Create a ``jobGroup``, which launches the specified job as the authenticated user.
Review Comment:
```suggestion
Create a ``jobGroup``, which launches the specified job as the authenticated user.
```
##########
airflow/providers/google/cloud/hooks/dataprep.py:
##########
@@ -121,8 +122,9 @@ def run_job_group(self, body_request: dict) -> dict[str, Any]:
"""
Creates a ``jobGroup``, which launches the specified job as the authenticated user.
This performs the same action as clicking on the Run Job button in the application.
+
Review Comment:
```suggestion
Creates a ``jobGroup``, which launches the specified job as the authenticated user.
This performs the same action as clicking on the Run Job button in the application.
```
##########
airflow/providers/google/cloud/operators/dataprep.py:
##########
@@ -66,7 +67,8 @@ class DataprepGetJobGroupOperator(GoogleCloudBaseOperator):
"""
Get the specified job group.
Review Comment:
```suggestion
Get the specified job group.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #31422: Add D400 pydocstyle check - Google provider only
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #31422:
URL: https://github.com/apache/airflow/pull/31422
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vincbeck commented on a diff in pull request #31422: Add D400 pydocstyle check - Google provider only
Posted by "vincbeck (via GitHub)" <gi...@apache.org>.
vincbeck commented on code in PR #31422:
URL: https://github.com/apache/airflow/pull/31422#discussion_r1202537734
##########
airflow/providers/google/cloud/utils/credentials_provider.py:
##########
@@ -141,10 +141,9 @@ def provide_gcp_conn_and_credentials(
) -> Generator[None, None, None]:
"""
Context manager that provides both:
-
- Google Cloud credentials for application supporting `Application Default Credentials (ADC)
strategy`__.
- - temporary value of :envvar:`AIRFLOW_CONN_GOOGLE_CLOUD_DEFAULT` connection
+ - temporary value of :envvar:`AIRFLOW_CONN_GOOGLE_CLOUD_DEFAULT` connection.
Review Comment:
What should be the change then?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a diff in pull request #31422: Add D400 pydocstyle check - Google provider only
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #31422:
URL: https://github.com/apache/airflow/pull/31422#discussion_r1201806674
##########
airflow/providers/google/cloud/log/stackdriver_task_handler.py:
##########
@@ -352,7 +351,7 @@ def get_external_log_url(self, task_instance: TaskInstance, try_number: int) ->
Creates an address for an external log collecting service.
:param task_instance: task instance object
:param try_number: task instance try_number to read logs from.
- :return: URL to the external log collection service
+ :return: URL to the external log collection service.
Review Comment:
Not right, should break before the first `:param:`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vincbeck commented on a diff in pull request #31422: Add D400 pydocstyle check - Google provider only
Posted by "vincbeck (via GitHub)" <gi...@apache.org>.
vincbeck commented on code in PR #31422:
URL: https://github.com/apache/airflow/pull/31422#discussion_r1202537734
##########
airflow/providers/google/cloud/utils/credentials_provider.py:
##########
@@ -141,10 +141,9 @@ def provide_gcp_conn_and_credentials(
) -> Generator[None, None, None]:
"""
Context manager that provides both:
-
- Google Cloud credentials for application supporting `Application Default Credentials (ADC)
strategy`__.
- - temporary value of :envvar:`AIRFLOW_CONN_GOOGLE_CLOUD_DEFAULT` connection
+ - temporary value of :envvar:`AIRFLOW_CONN_GOOGLE_CLOUD_DEFAULT` connection.
Review Comment:
What should be the change then?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a diff in pull request #31422: Add D400 pydocstyle check - Google provider only
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #31422:
URL: https://github.com/apache/airflow/pull/31422#discussion_r1201805797
##########
airflow/providers/google/cloud/operators/bigquery.py:
##########
@@ -2509,8 +2511,8 @@ def execute(self, context: Context):
class BigQueryInsertJobOperator(GoogleCloudBaseOperator):
"""
Executes a BigQuery job. Waits for the job to complete and returns job id.
- This operator work in the following way:
+ This operator work in the following way:
- it calculates a unique hash of the job using job's configuration or uuid if ``force_rerun`` is True
Review Comment:
I think this would break list rendering. An empty line is needed after the `:` as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vincbeck commented on pull request #31422: Add D400 pydocstyle check - Google provider only
Posted by "vincbeck (via GitHub)" <gi...@apache.org>.
vincbeck commented on PR #31422:
URL: https://github.com/apache/airflow/pull/31422#issuecomment-1567247359
@uranusjr any chance you can review it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a diff in pull request #31422: Add D400 pydocstyle check - Google provider only
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #31422:
URL: https://github.com/apache/airflow/pull/31422#discussion_r1201796492
##########
airflow/providers/google/cloud/triggers/bigquery.py:
##########
@@ -549,7 +549,7 @@ class BigQueryTablePartitionExistenceTrigger(BigQueryTableExistenceTrigger):
:param table_id: The table ID of the requested table.
:param gcp_conn_id: Reference to google cloud connection id
:param hook_params: params for hook
- :param poll_interval: polling period in seconds to check for the status
+ :param poll_interval: polling period in seconds to check for the status.
Review Comment:
Should add newline before the first `:param:` instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a diff in pull request #31422: Add D400 pydocstyle check - Google provider only
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #31422:
URL: https://github.com/apache/airflow/pull/31422#discussion_r1201793449
##########
airflow/providers/google/cloud/utils/credentials_provider.py:
##########
@@ -141,10 +141,9 @@ def provide_gcp_conn_and_credentials(
) -> Generator[None, None, None]:
"""
Context manager that provides both:
-
- Google Cloud credentials for application supporting `Application Default Credentials (ADC)
strategy`__.
- - temporary value of :envvar:`AIRFLOW_CONN_GOOGLE_CLOUD_DEFAULT` connection
+ - temporary value of :envvar:`AIRFLOW_CONN_GOOGLE_CLOUD_DEFAULT` connection.
Review Comment:
I think this would break the item list rendering…
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vincbeck commented on pull request #31422: Add D400 pydocstyle check - Google provider only
Posted by "vincbeck (via GitHub)" <gi...@apache.org>.
vincbeck commented on PR #31422:
URL: https://github.com/apache/airflow/pull/31422#issuecomment-1572098211
Someone to merge this one please? :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org