You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/05/03 16:14:18 UTC
[GitHub] [airflow] hubert-pietron opened a new pull request, #23452: Fix closing connection dbapi.get_pandas_df
hubert-pietron opened a new pull request, #23452:
URL: https://github.com/apache/airflow/pull/23452
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #22947
related: #22947
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
closes: #22947
related: #22947
Using **dbapi.get_pandas_df** with parameter _chunksize_ caused error, because function returned generator that reference to closed connection. **dbapi.get_pandas_df_by_chunks** allow to iterate through that generator before connection is closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a diff in pull request #23452: Fix closing connection dbapi.get_pandas_df
Posted by GitBox <gi...@apache.org>.
uranusjr commented on code in PR #23452:
URL: https://github.com/apache/airflow/pull/23452#discussion_r864022730
##########
airflow/hooks/dbapi.py:
##########
@@ -128,6 +131,15 @@ def get_pandas_df(self, sql, parameters=None, **kwargs):
with closing(self.get_conn()) as conn:
return psql.read_sql(sql, con=conn, params=parameters, **kwargs)
+ def get_pandas_df_by_chunks(self, sql, parameters=None, **kwargs):
Review Comment:
Let’s make this
```suggestion
def get_pandas_df_by_chunks(self, sql, parameters=None, *, chunksize, **kwargs):
```
So this function is directly usable as public API and a user won’t pass incorrect arguments in.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #23452: Fix closing connection dbapi.get_pandas_df
Posted by GitBox <gi...@apache.org>.
potiuk merged PR #23452:
URL: https://github.com/apache/airflow/pull/23452
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #23452: Fix closing connection dbapi.get_pandas_df
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #23452:
URL: https://github.com/apache/airflow/pull/23452#issuecomment-1135362751
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hubert-pietron commented on a diff in pull request #23452: Fix closing connection dbapi.get_pandas_df
Posted by GitBox <gi...@apache.org>.
hubert-pietron commented on code in PR #23452:
URL: https://github.com/apache/airflow/pull/23452#discussion_r866736639
##########
airflow/hooks/dbapi.py:
##########
@@ -128,6 +131,15 @@ def get_pandas_df(self, sql, parameters=None, **kwargs):
with closing(self.get_conn()) as conn:
return psql.read_sql(sql, con=conn, params=parameters, **kwargs)
+ def get_pandas_df_by_chunks(self, sql, parameters=None, **kwargs):
Review Comment:
Thank You for checking, I think it's good idea. I will make changes in PR tomorrow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org