You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/01/14 11:53:50 UTC

[GitHub] [airflow] lassanouer opened a new issue #13672: Dags are not able to get connections via hashicorp_vault

lassanouer opened a new issue #13672:
URL: https://github.com/apache/airflow/issues/13672


   Using Hashicorp_vault was a good idea to make things safe and easy for airflow users,
   
   my airflow version: 1.10.14
   
   **Use case:**
   Create a secret X in vault, config airflow to get that connections
   Create a 2 dags,
       
   
   - in first one, u call the connection X that you create in vault
   - the second one, call a connection Y that is not in vault
   
   launch airflow,
   
   ==> In user interface, there is only error to get Y connection, and when i launch the dag :
   get_connections raise AirflowException("The conn_id `{0}` isn't defined".format(conn_id))
   
   i tryed debugge that and i found out that if i call get_conn_uri from hashicorp_vault i got the connection... so the real probleme is in airflow.secrets calling secretes
   
   
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #13672: Dags are not able to get connections via hashicorp_vault

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #13672:
URL: https://github.com/apache/airflow/issues/13672#issuecomment-760149327


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] kaxil commented on issue #13672: Dags are not able to get connections via hashicorp_vault

Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #13672:
URL: https://github.com/apache/airflow/issues/13672#issuecomment-760530190


   The error is genuine though, have you added a connection Y in either Metadata DB or as Environment Variable.
   
   If no-one of this, then it will error


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] lassanouer commented on issue #13672: Dags are not able to get connections via hashicorp_vault

Posted by GitBox <gi...@apache.org>.
lassanouer commented on issue #13672:
URL: https://github.com/apache/airflow/issues/13672#issuecomment-760572619


   my issue is when i use vault for connections, it doesn't work.
   the connection Y case is to show that from the user interface, airflow is recognizing the connection X and doesn't show error for it.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] kaxil commented on issue #13672: Dags are not able to get connections via hashicorp_vault

Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #13672:
URL: https://github.com/apache/airflow/issues/13672#issuecomment-797575236


   I wasn't able to reproduce this, please try it with 2.0.1 -- We only backport critical fixes to 1.10.x now


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] kaxil closed issue #13672: Dags are not able to get connections via hashicorp_vault

Posted by GitBox <gi...@apache.org>.
kaxil closed issue #13672:
URL: https://github.com/apache/airflow/issues/13672


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] lassanouer commented on issue #13672: Dags are not able to get connections via hashicorp_vault

Posted by GitBox <gi...@apache.org>.
lassanouer commented on issue #13672:
URL: https://github.com/apache/airflow/issues/13672#issuecomment-760578827


   Yeah,
   
   1) create a secret X in vault in secrets/somewhere/in/vault for exemple
   
   2) add this env var to airflow:
    - name: AIRFLOW__SECRETS__BACKEND
       value: "airflow.contrib.secrets.hashicorp_vault.VaultBackend"
     - name: AIRFLOW__SECRETS__BACKEND_KWARGS
       value: "{\"connections_path\": \"somewhere/in/vault\", \"mount_point\":\"secrets\", \"url\":\"https://vault.xxx.app\", \"role_id\":\"xxxxxxxxxx\", \"secret_id\":\"xxxxxxxxx\", \"auth_type\":\"approle\"}"
   
   3) create a dag where you call this connection X in HttpSensor for example
   
    
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] kaxil commented on issue #13672: Dags are not able to get connections via hashicorp_vault

Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #13672:
URL: https://github.com/apache/airflow/issues/13672#issuecomment-762425919


   If possible, can you please also check with Airflow 2.0 ? and see if you can reproduce this


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] kaxil commented on issue #13672: Dags are not able to get connections via hashicorp_vault

Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #13672:
URL: https://github.com/apache/airflow/issues/13672#issuecomment-760575454


   > my issue is when i use vault for connections, it doesn't work.
   > the connection Y case is to show that from the user interface, airflow is recognizing the connection X and doesn't show error for it.
   
   Can you post the steps to reproduce it?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org