You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "stroeovidiu (via GitHub)" <gi...@apache.org> on 2023/06/07 11:12:39 UTC
[GitHub] [airflow] stroeovidiu opened a new issue, #31759: User AWS postgresql with IAM auth as backend
stroeovidiu opened a new issue, #31759:
URL: https://github.com/apache/airflow/issues/31759
### Apache Airflow version
2.6.1
### What happened
I am trying to deploy airflow in aws eks ( kubernetes ) using the helm chart.
I am trying to set as external database an aws rds postgresql with IAM auth enabled.
I have created the IAM role and the corresponding service account but it doesn't connect.
I get "port 6432 failed: FATAL: PAM authentication failed "
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.html
Does airflow support aws rds postgres as backend using iam auth ?
### What you think should happen instead
_No response_
### How to reproduce
use an rds postgresql with iam auth as backend
### Operating System
kubernetes
### Versions of Apache Airflow Providers
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hussein-awala commented on issue #31759: User AWS postgresql with IAM auth as backend
Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala commented on issue #31759:
URL: https://github.com/apache/airflow/issues/31759#issuecomment-1583329639
Could you update the issue by providing more information about your Airflow configurations, the error/problem you have and a full example to reproduce the issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on issue #31759: User AWS postgresql with IAM auth as backend
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #31759:
URL: https://github.com/apache/airflow/issues/31759#issuecomment-1635078984
This issue has been automatically marked as stale because it has been open for 30 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on issue #31759: User AWS postgresql with IAM auth as backend
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #31759:
URL: https://github.com/apache/airflow/issues/31759#issuecomment-1687225454
This issue has been closed because it has not received response from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] stroeovidiu commented on issue #31759: User AWS postgresql with IAM auth as backend
Posted by "stroeovidiu (via GitHub)" <gi...@apache.org>.
stroeovidiu commented on issue #31759:
URL: https://github.com/apache/airflow/issues/31759#issuecomment-1588694965
It is a vanilla airflow deployed in eks using as backend a postgresql database in RDS.
As for the custom configs:
- an IAM user for the postgresql DB
- an IAM role with permissions to connect to that db with that iam role
- and kubernetes service account ( created using the helm provided values ) with annotations to map to the above IAM role.
What i am expecting is that the airflow pods/containers start but they get a PAM auth error .
Thank you !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] stroeovidiu commented on issue #31759: User AWS postgresql with IAM auth as backend
Posted by "stroeovidiu (via GitHub)" <gi...@apache.org>.
stroeovidiu commented on issue #31759:
URL: https://github.com/apache/airflow/issues/31759#issuecomment-1635373293
my issue is connecting to postgres using IAM Auth not password auth. The link you provided has no info on the matter i am trying to fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] closed issue #31759: User AWS postgresql with IAM auth as backend
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #31759: User AWS postgresql with IAM auth as backend
URL: https://github.com/apache/airflow/issues/31759
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #31759: User AWS postgresql with IAM auth as backend
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #31759:
URL: https://github.com/apache/airflow/issues/31759#issuecomment-1580559348
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on issue #31759: User AWS postgresql with IAM auth as backend
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #31759:
URL: https://github.com/apache/airflow/issues/31759#issuecomment-1676664065
This issue has been automatically marked as stale because it has been open for 30 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vincbeck commented on issue #31759: User AWS postgresql with IAM auth as backend
Posted by "vincbeck (via GitHub)" <gi...@apache.org>.
vincbeck commented on issue #31759:
URL: https://github.com/apache/airflow/issues/31759#issuecomment-1590045847
Did you follow the recommandations as mentioned in the [documentation](https://airflow.apache.org/docs/apache-airflow/stable/howto/set-up-database.html#setting-up-a-postgresql-database). Specifically:
> For managed Postgres such as Redshift, Azure Postgresql, CloudSQL, Amazon RDS, you should use keepalives_idle in the connection parameters and set it to less than the idle time because those services will close idle connections after some time of inactivity (typically 300 seconds), which results with error The error: psycopg2.operationalerror: SSL SYSCALL error: EOF detected. The keepalive settings can be changed via sql_alchemy_connect_args configuration parameter [Configuration Reference](https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html) in [database] section. You can configure the args for example in your local_settings.py and the sql_alchemy_connect_args should be a full import path to the dictionary that stores the configuration parameters. You can read about [Postgres Keepalives](https://www.postgresql.org/docs/current/libpq-connect.html). An example setup for keepalives that has been observed to fix the problem might be:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org