You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "shakeelansari63 (via GitHub)" <gi...@apache.org> on 2023/08/02 07:11:17 UTC
[GitHub] [airflow] shakeelansari63 opened a new issue, #33025: Problem with Microsoft Azure Provider base_azure hook
shakeelansari63 opened a new issue, #33025:
URL: https://github.com/apache/airflow/issues/33025
### Apache Airflow version
2.6.3
### What happened
When using AzureBaseHook from airflow/providers/microsoft/azure/hooks/base_azure.py with SDKClient class, the get_conn() method fails to instanciate the Client class object.
The error message says, `missing parameter credential`
This is because the sdk_client is being instanciated with `credentials=ServicePrincipalCredentials`
Notice an extra `s` in credentials key which is not needed.
### What you think should happen instead
The sdk_client should be instanciated with `credential=ServicePrincipalCredential` which will be instanciated correctly
### How to reproduce
1. Install any Azure client, let's say azure-monitor-query
`pip install azure-monitor-query`
2. Use the Azure connection with Azure client
```
from airflow.providers.microsoft.azure.hoojs.base_azure import AzureBaseHook
from azure.monitor.query import LogsQueryClient
# Lets say we have an azure connection with name Azure_Default
conn = AzureBaseHook(sdk_client = LogsQueryClient, conn_id = 'Azure_Default')
client = conn.get_conn()
```
This will fail -
### Operating System
Any
### Versions of Apache Airflow Providers
apache-airflo-providers-microsoft-azure v6.2.1
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #33025: Problem with Microsoft Azure Provider base_azure hook
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #33025:
URL: https://github.com/apache/airflow/issues/33025#issuecomment-1661631031
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] shakeelansari63 commented on issue #33025: Problem with Microsoft Azure Provider base_azure hook
Posted by "shakeelansari63 (via GitHub)" <gi...@apache.org>.
shakeelansari63 commented on issue #33025:
URL: https://github.com/apache/airflow/issues/33025#issuecomment-1663700599
@nathadfield , I got the issue here.
Looks like you are using very old version of azure SDK.
I looked into one of the failure, for ContainerInstanceManagementClient.
This is from `provider.yaml`
```
- azure-mgmt-container-instance>=1.5.0,<2.0
```
This is very old version of azure SDK (from 2019) which is still using `credentials` parameter. But in newer version all azure sdks use `credential` parameter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] nathadfield commented on issue #33025: Problem with Microsoft Azure Provider base_azure hook
Posted by "nathadfield (via GitHub)" <gi...@apache.org>.
nathadfield commented on issue #33025:
URL: https://github.com/apache/airflow/issues/33025#issuecomment-1663713162
@shakeelansari63 Yep. I'll ask around to see what I can find out but it seems like you won't be able to use `LogsQueryClient` until `azure-mgmt-containerinstance` is brought up to date.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pankajkoti commented on issue #33025: Problem with Microsoft Azure Provider base_azure hook
Posted by "pankajkoti (via GitHub)" <gi...@apache.org>.
pankajkoti commented on issue #33025:
URL: https://github.com/apache/airflow/issues/33025#issuecomment-1663699871
I added a comment that could be the cause https://github.com/apache/airflow/pull/33040#issuecomment-1663696347 on the PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #33025: Problem with Microsoft Azure Provider base_azure hook
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on issue #33025:
URL: https://github.com/apache/airflow/issues/33025#issuecomment-1663723160
I'm closing this issue as the root cause is updating old package as listed in https://github.com/apache/airflow/issues/30199
PRs are welcome.
Azure offers [Airflow as managed service in their eco system](https://techcommunity.microsoft.com/t5/azure-data-factory-blog/introducing-managed-airflow-in-azure-data-factory/ba-p/3730151) try maybe contacting Azure support maybe they will assign an engineer to work on Azure open source issues.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] nathadfield closed issue #33025: Problem with Microsoft Azure Provider base_azure hook
Posted by "nathadfield (via GitHub)" <gi...@apache.org>.
nathadfield closed issue #33025: Problem with Microsoft Azure Provider base_azure hook
URL: https://github.com/apache/airflow/issues/33025
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal closed issue #33025: Problem with Microsoft Azure Provider base_azure hook
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal closed issue #33025: Problem with Microsoft Azure Provider base_azure hook
URL: https://github.com/apache/airflow/issues/33025
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] nathadfield commented on issue #33025: Problem with Microsoft Azure Provider base_azure hook
Posted by "nathadfield (via GitHub)" <gi...@apache.org>.
nathadfield commented on issue #33025:
URL: https://github.com/apache/airflow/issues/33025#issuecomment-1663716531
#30199
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] nathadfield commented on issue #33025: Problem with Microsoft Azure Provider base_azure hook
Posted by "nathadfield (via GitHub)" <gi...@apache.org>.
nathadfield commented on issue #33025:
URL: https://github.com/apache/airflow/issues/33025#issuecomment-1661856346
@shakeelansari63 This looks like a simple change but I have not been able to replicate this with the code you provided. Can you review this or do you have an example DAG that can illustrate the problem?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] nathadfield commented on issue #33025: Problem with Microsoft Azure Provider base_azure hook
Posted by "nathadfield (via GitHub)" <gi...@apache.org>.
nathadfield commented on issue #33025:
URL: https://github.com/apache/airflow/issues/33025#issuecomment-1663694312
Changing this seems to break `AzureContainerInstanceHook`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] nathadfield commented on issue #33025: Problem with Microsoft Azure Provider base_azure hook
Posted by "nathadfield (via GitHub)" <gi...@apache.org>.
nathadfield commented on issue #33025:
URL: https://github.com/apache/airflow/issues/33025#issuecomment-1663591920
@shakeelansari63 Having made this change, we are now encountering [test failures](https://github.com/apache/airflow/actions/runs/5740945880/job/15560093916?pr=33040) and so I remain unconvinced that this is indeed a problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] shakeelansari63 commented on issue #33025: Problem with Microsoft Azure Provider base_azure hook
Posted by "shakeelansari63 (via GitHub)" <gi...@apache.org>.
shakeelansari63 commented on issue #33025:
URL: https://github.com/apache/airflow/issues/33025#issuecomment-1663703912
From what I can see from Pypi, latest version of `azure-mgmt-containerinstance` is `10.1.0`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org