You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/12/21 14:32:04 UTC

[GitHub] [airflow] juditkuti opened a new issue, #28524: Use only parameters for client_type that are required

juditkuti opened a new issue, #28524:
URL: https://github.com/apache/airflow/issues/28524

   ### Description
   
   Currently we see an error whenever the dw-redshift airflow connection is called in our DAGs. It doesn't block running the DAGs but it would be better to have less noisy logs. This issue is due to File "/usr/local/lib/python3.9/site-packages/airflow/providers/postgres/hooks/postgres.py", line 113, in get_conn self.conn = psycopg2.connect(**conn_args) which does not have client_type removed as a connection parameter. The code should rather behave like this:
   ```
           for arg_name, arg_val in conn.extra_dejson.items():
               if arg_name not in [
                   "iam",
                   "redshift",
                   "cursor",
                   "cluster-identifier",
                   "aws_conn_id",
                   "client_type",
               ]:
                   conn_args[arg_name] = arg_val
   ```
   
   
   ### Use case/motivation
   
   Error logs should go away in the DAG Airflow logs:
   ```
   Traceback (most recent call last):
     File "/usr/local/lib/python3.9/site-packages/openlineage/airflow/extractors/manager.py", line 48, in extract_metadata
       task_metadata = extractor.extract()
     File "/usr/local/lib/python3.9/site-packages/openlineage/airflow/extractors/sql_extractor.py", line 62, in extract
       inputs, outputs = get_table_schemas(
     File "/usr/local/lib/python3.9/site-packages/openlineage/airflow/extractors/dbapi_utils.py", line 44, in get_table_schemas
       with closing(hook.get_conn()) as conn:
     File "/usr/local/lib/python3.9/site-packages/airflow/providers/postgres/hooks/postgres.py", line 113, in get_conn
       self.conn = psycopg2.connect(**conn_args)
     File "/usr/local/lib/python3.9/site-packages/psycopg2/__init__.py", line 121, in connect
       dsn = _ext.make_dsn(dsn, **kwargs)
     File "/usr/local/lib/python3.9/site-packages/psycopg2/extensions.py", line 167, in make_dsn
       parse_dsn(dsn)
   psycopg2.ProgrammingError: invalid dsn: invalid connection option "client_type"
   ```
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] github-actions[bot] closed issue #28524: Use only parameters for client_type that are required

Posted by github-actions.
github-actions[bot] closed issue #28524: Use only parameters for client_type that are required
URL: https://github.com/apache/airflow/issues/28524


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] github-actions[bot] commented on issue #28524: Use only parameters for client_type that are required

Posted by github-actions.
github-actions[bot] commented on issue #28524:
URL: https://github.com/apache/airflow/issues/28524#issuecomment-1407208134

   This issue has been closed because it has not received response from the issue author.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] github-actions[bot] commented on issue #28524: Use only parameters for client_type that are required

Posted by github-actions.
github-actions[bot] commented on issue #28524:
URL: https://github.com/apache/airflow/issues/28524#issuecomment-1399092083

   This issue has been automatically marked as stale because it has been open for 30 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] Taragolis commented on issue #28524: Use only parameters for client_type that are required

Posted by GitBox <gi...@apache.org>.
Taragolis commented on issue #28524:
URL: https://github.com/apache/airflow/issues/28524#issuecomment-1361412320

   @juditkuti could you clarify is this issue related to [PostgreSQL Connection](https://airflow.apache.org/docs/apache-airflow-providers-postgres/stable/connections/postgres.html ) from postgres-provider or [Amazon Redshift Connection](https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/connections/redshift.html) from amazon-provider?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] boring-cyborg[bot] commented on issue #28524: Use only parameters for client_type that are required

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #28524:
URL: https://github.com/apache/airflow/issues/28524#issuecomment-1361386176

   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org