You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/08/24 12:38:29 UTC
[GitHub] [airflow] agivola opened a new issue #10505: PostgressHook connection - pass additional extra paramters
agivola opened a new issue #10505:
URL: https://github.com/apache/airflow/issues/10505
Hi, when working with PostgressHook connection and use it in multi-threads we need to pass extra parameters like pool_size, max_overflow and max_timeout as connection parameters . Now only this parameters are accepted:
'sslmode', 'sslcert', 'sslkey','sslrootcert', 'sslcrl', 'application_name','keepalives_idle'
https://airflow.apache.org/docs/stable/_modules/airflow/hooks/postgres_hook.html
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] RosterIn commented on issue #10505: PostgressHook connection - pass additional extra paramters
Posted by GitBox <gi...@apache.org>.
RosterIn commented on issue #10505:
URL: https://github.com/apache/airflow/issues/10505#issuecomment-694836845
@daemon-demon There was a PR that tried to achieve that https://github.com/apache/airflow/pull/7632
maybe it will help you as a starting point
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] agivola commented on issue #10505: PostgressHook connection - pass additional extra paramters
Posted by GitBox <gi...@apache.org>.
agivola commented on issue #10505:
URL: https://github.com/apache/airflow/issues/10505#issuecomment-694859480
@kbobrien Hi, I think that the best way is to allow pass all parameters via extra_json without restrictions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kbobrien commented on issue #10505: PostgressHook connection - pass additional extra paramters
Posted by GitBox <gi...@apache.org>.
kbobrien commented on issue #10505:
URL: https://github.com/apache/airflow/issues/10505#issuecomment-687239454
I was about to open a ticket to support the `target_session_attrs` psycopg2 parameter but this ticket could cover that use case too. Is there really a need to limit the parameters supplied to `psycopg2.connect()` on the Airflow side, psycopg2 itself doesn't limit the parameters it passes through to the underlying library?
Could I suggest that we remove the List of parameters supported by the Hook altogether and just pass through what users supply in the extra_json as kv's? Since this check doesn't fail hard it provides limited benefit to users who probably expect their extra parameters were passed down since they got no warnings or errors.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb closed issue #10505: PostgressHook connection - pass additional extra paramters
Posted by GitBox <gi...@apache.org>.
ashb closed issue #10505:
URL: https://github.com/apache/airflow/issues/10505
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #10505: PostgressHook connection - pass additional extra paramters
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #10505:
URL: https://github.com/apache/airflow/issues/10505#issuecomment-679101304
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] daemon-demon commented on issue #10505: PostgressHook connection - pass additional extra paramters
Posted by GitBox <gi...@apache.org>.
daemon-demon commented on issue #10505:
URL: https://github.com/apache/airflow/issues/10505#issuecomment-694995529
@RosterIn @agivola
I will be creating a PR today for the fix I put in to allow all parameters to pass without restrictions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj edited a comment on issue #10505: PostgressHook connection - pass additional extra paramters
Posted by GitBox <gi...@apache.org>.
mik-laj edited a comment on issue #10505:
URL: https://github.com/apache/airflow/issues/10505#issuecomment-679117807
This is a great ticket for a first contribution. Would you like to help us with this?
Here is the development environment guide: https://github.com/apache/airflow/blob/master/BREEZE.rst
Here is the contributor guide: https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst
If you have problems, I will be happy to help.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #10505: PostgressHook connection - pass additional extra paramters
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #10505:
URL: https://github.com/apache/airflow/issues/10505#issuecomment-679392943
@daemon-demon I assigned you to this ticket.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #10505: PostgressHook connection - pass additional extra paramters
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #10505:
URL: https://github.com/apache/airflow/issues/10505#issuecomment-679117807
This is a great ticket for a first contribution. Would you like to help us with this?
Here is the development environment guide: https://github.com/apache/airflow/blob/master/BREEZE.rst
Here is the contributor guide: https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] daemon-demon commented on issue #10505: PostgressHook connection - pass additional extra paramters
Posted by GitBox <gi...@apache.org>.
daemon-demon commented on issue #10505:
URL: https://github.com/apache/airflow/issues/10505#issuecomment-679391718
Hi, I would like to work on this as my first contribution.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org