You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "manmeetkaur (via GitHub)" <gi...@apache.org> on 2023/06/09 06:40:56 UTC
[GitHub] [airflow] manmeetkaur opened a new issue, #31811: Airflow Connection Type Azure Blob Storage - Shared Access Key field
manmeetkaur opened a new issue, #31811:
URL: https://github.com/apache/airflow/issues/31811
### Apache Airflow version
Other Airflow 2 version (please specify below)
### What happened
Airflow Version 2.5.2
Issue: Create a connection of type azure blob storage using the method #3 described in [OSS docs](https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/connections/wasb.html).
When we use the storage account name and the access key as in the below screenshot, the connection works just fine.
<img width="637" alt="connection-blob-storage-access-key" src="https://github.com/apache/airflow/assets/75730393/6cf76b44-f65f-40c0-8279-32c58b6d57ba">
Then what is the purpose of the extra field called `Blob Storage Shared Access Key(Optional)`? When I tried to put the access key in this field, then connection fails with the below error on testing:
```
invalid url http://
```
Code reference: https://github.com/apache/airflow/blob/main/airflow/providers/microsoft/azure/hooks/wasb.py#L190-L192
### What you think should happen instead
_No response_
### How to reproduce
- Create a Azure Storage account
- Go to Access Control (IAM), copy the key
- Spin up new local airflow environment using Astro CLI with runtime version 7.4.1
- Go to Airflow UI -> Admin -> Connections -> create a new connection of type Azure blob Storage
- Enter the name of the storage account in Blob Storage Login and the key copied from Azure to Blob Shared Access Key (Optional) and click on Test connection.
### Operating System
Astro runtime 7.4.1 image on MacOS
### Versions of Apache Airflow Providers
_No response_
### Deployment
Astronomer
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #31811: Airflow Connection Type Azure Blob Storage - Shared Access Key field
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #31811:
URL: https://github.com/apache/airflow/issues/31811#issuecomment-1584444675
feel free.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Adaverse commented on issue #31811: Airflow Connection Type Azure Blob Storage - Shared Access Key field
Posted by "Adaverse (via GitHub)" <gi...@apache.org>.
Adaverse commented on issue #31811:
URL: https://github.com/apache/airflow/issues/31811#issuecomment-1584350824
Will take this up!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #31811: Airflow Connection Type Azure Blob Storage - Shared Access Key field
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #31811: Airflow Connection Type Azure Blob Storage - Shared Access Key field
URL: https://github.com/apache/airflow/issues/31811
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Adaverse commented on issue #31811: Airflow Connection Type Azure Blob Storage - Shared Access Key field
Posted by "Adaverse (via GitHub)" <gi...@apache.org>.
Adaverse commented on issue #31811:
URL: https://github.com/apache/airflow/issues/31811#issuecomment-1600337711
> Then what is the purpose of the extra field called Blob Storage Shared Access Key(Optional)?
To answer above question, we neeed to look into the signature of `BlobServiceClient` -
`BlobServiceClient(account_url: str, credential: str | Dict[str, str] | AzureNamedKeyCredential | AzureSasCredential | TokenCredential | None = None, **kwargs: Any)`
We can see account_url is required but credential is optional. The reason why credentail is optional because the account_url can itself contain the credential (example SAS). Since you are filling the `shared access key` field, it expects the `account_url` field to be filled as well of the form `https://{account_name}.blob.core.windows.net` and since this field is blank in your case hence the error. The `Blob Storage Login` is completly ignored in this case.
The first case works just fine because URL is being built [here](https://github.com/apache/airflow/blob/fe7a1aa0962538a99f5e73dbd3890a6f210d22ed/airflow/providers/microsoft/azure/hooks/wasb.py#L217) using `Blob Storage Login`
Will create a PR to handle this case. But I hope it clears the doubt here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org