You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "hussein-awala (via GitHub)" <gi...@apache.org> on 2023/04/10 01:55:19 UTC
[GitHub] [airflow] hussein-awala opened a new pull request, #30558: [WIP] Clean microsoft azure provider by deleting the custom prefix from conn extra fields
hussein-awala opened a new pull request, #30558:
URL: https://github.com/apache/airflow/pull/30558
closes: #30557
---
In this PR I fix the UI fields name by removing the custom prefix from Azure conn extra fields, deprecating the used fields with this prefix in order to remove them in the future, clean the documentation, and clean the provider tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #30558: [WIP] Clean microsoft azure provider by deleting the custom prefix from conn extra fields
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30558:
URL: https://github.com/apache/airflow/pull/30558#issuecomment-1567613148
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #30558: Clean microsoft azure provider by deleting the custom prefix from conn extra fields
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #30558:
URL: https://github.com/apache/airflow/pull/30558
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #30558: [WIP] Clean microsoft azure provider by deleting the custom prefix from conn extra fields
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #30558:
URL: https://github.com/apache/airflow/pull/30558#issuecomment-1508998886
After defining connection with 2.3 of course.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hussein-awala commented on pull request #30558: Clean microsoft azure provider by deleting the custom prefix from conn extra fields
Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala commented on PR #30558:
URL: https://github.com/apache/airflow/pull/30558#issuecomment-1668523646
@potiuk I tested this PR with 2.4.0 (the current min version) and with main, I can create connections with and without the `extra__azure__` prefix without any issue. What kind of tests should I do before merging it?
And for this commet:
> Hmm. I think the problem was that without the "ensure prefixes" providers_manager woudl crash during building the UI fir connections or at least wreak some havoc. I think we would need to build such provder and install it in airflow 2.3 to be sure.
There is no problems with the UI:



(6.2.3 because I built it with breeze and installed it from source in a clean conda env)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #30558: Clean microsoft azure provider by deleting the custom prefix from conn extra fields
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #30558:
URL: https://github.com/apache/airflow/pull/30558#issuecomment-1668540056
I can't remember exactly (the comment was from more than 3 months ago) but I believe it was based on "providers back-compat tests" failing back then. We moved to 2.4 min version since, so I think this is good to go now - maybe because of back-compat problem going away because of the min-airflow version bump :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #30558: [WIP] Clean microsoft azure provider by deleting the custom prefix from conn extra fields
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #30558:
URL: https://github.com/apache/airflow/pull/30558#issuecomment-1508998293
Hmm. I think the problem was that without the "ensure prefixes" providers_manager woudl crash during building the UI fir connections or at least wreak some havoc. I think we would need to build such provder and install it in airflow 2.3 to be sure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #30558: [WIP] Clean microsoft azure provider by deleting the custom prefix from conn extra fields
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on PR #30558:
URL: https://github.com/apache/airflow/pull/30558#issuecomment-1501422411
We also have entry for Azure in https://github.com/apache/airflow/issues/28790
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hussein-awala commented on pull request #30558: [WIP] Clean microsoft azure provider by deleting the custom prefix from conn extra fields
Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala commented on PR #30558:
URL: https://github.com/apache/airflow/pull/30558#issuecomment-1507770774
The method `_ensure_prefixes` was added in #27041 by @dstandish:
https://github.com/apache/airflow/blob/058b6ebdd863ff9178007f5750cc4cde4d3c463f/airflow/providers/microsoft/azure/hooks/fileshare.py#L29-L33
I don't think we really need to wait until bumping the provider min airflow version to >= 2.5.0, since we check for the two variables (with and without `extra_...`) in each extra field loading, and with this PR, we will have:
- For the new connections, the extra fields will be created without `extra_` prefix -> this should work without any problem
- For the connection which have fields with `extra_`, when we update them will have a new field without the `extra_` prefix and another one with `extra_` prefix but with the old value, this old value will not be processed as long as the new one is present in the extra.
- For the connection which have fields with `extra_`, without any update, the provider will use the fields with the prefix, and a warning will be logged to prompt the user to update the connection extra fields.
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #30558: [WIP] Clean microsoft azure provider by deleting the custom prefix from conn extra fields
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30558:
URL: https://github.com/apache/airflow/pull/30558#issuecomment-1663120663
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org