You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/04/02 14:25:47 UTC
[GitHub] [airflow] sunkickr opened a new pull request #15159: Add dynamic connection fields to Azure Connection
sunkickr opened a new pull request #15159:
URL: https://github.com/apache/airflow/pull/15159
This PR adds custom connection form widgets and behavior to the Azure Connection defined in the AzureBaseHook.
screenshot:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a change in pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
josh-fell commented on a change in pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#discussion_r609747014
##########
File path: airflow/providers/microsoft/azure/hooks/azure_container_instance.py
##########
@@ -41,11 +41,53 @@ class AzureContainerInstanceHook(AzureBaseHook):
:type conn_id: str
"""
- conn_name_attr = 'conn_id'
+ conn_name_attr = 'azure_conn_id'
default_conn_name = 'azure_default'
conn_type = 'azure_container_instances'
hook_name = 'Azure Container Instance'
+ @staticmethod
+ def get_connection_form_widgets() -> Dict[str, Any]:
+ """Returns connection widgets to add to connection form"""
+ from flask_appbuilder.fieldwidgets import BS3TextFieldWidget
+ from flask_babel import lazy_gettext
+ from wtforms import StringField
+
+ return {
+ "extra__azure__tenantId": StringField(
+ lazy_gettext('Azure Tenant ID'), widget=BS3TextFieldWidget()
+ ),
+ "extra__azure__subscriptionId": StringField(
+ lazy_gettext('Azure Subscription ID'), widget=BS3TextFieldWidget()
+ ),
+ }
+
+ @staticmethod
+ def get_ui_field_behaviour() -> Dict:
+ """Returns custom field behaviour"""
+ import json
+
+ return {
+ "hidden_fields": ['schema', 'port', 'host'],
+ "relabeling": {
+ 'login': 'Azure Client ID',
+ 'password': 'Azure Secret',
+ },
+ "placeholders": {
+ 'extra': json.dumps(
+ {
+ "key_path": "path to json file for auth",
+ "key_json": "specifies json dict for auth",
+ },
+ indent=1,
+ ),
+ 'login': 'client_id (token credentials auth)',
+ 'password': 'secret (token credentials auth)',
+ 'extra__azure__tenantId': 'tenentId (token credentials auth)',
Review comment:
```suggestion
'extra__azure__tenantId': 'TenantId (token credentials auth)',
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#issuecomment-814483963
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a change in pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
josh-fell commented on a change in pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#discussion_r609769851
##########
File path: airflow/providers/microsoft/azure/hooks/azure_data_lake.py
##########
@@ -49,6 +49,39 @@ class AzureDataLakeHook(BaseHook):
conn_type = 'azure_data_lake'
hook_name = 'Azure Data Lake'
+ @staticmethod
+ def get_connection_form_widgets() -> Dict[str, Any]:
+ """Returns connection widgets to add to connection form"""
+ from flask_appbuilder.fieldwidgets import BS3TextFieldWidget
+ from flask_babel import lazy_gettext
+ from wtforms import StringField
+
+ return {
+ "extra__azure_data_lake__tenant": StringField(
+ lazy_gettext('Azure Tenant ID'), widget=BS3TextFieldWidget()
+ ),
+ "extra__azure_data_lake__account_name": StringField(
+ lazy_gettext('Azure DataLake Store Name'), widget=BS3TextFieldWidget()
+ ),
+ }
+
+ @staticmethod
+ def get_ui_field_behaviour() -> Dict:
+ """Returns custom field behaviour"""
+ return {
+ "hidden_fields": ['schema', 'port', 'host', 'extra'],
+ "relabeling": {
+ 'login': 'Azure Client ID',
+ 'password': 'Azure Client Secret',
+ },
+ "placeholders": {
+ 'login': 'client_id',
Review comment:
```suggestion
'login': 'client id',
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on a change in pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on a change in pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#discussion_r608875129
##########
File path: airflow/providers/microsoft/azure/hooks/wasb.py
##########
@@ -57,6 +57,50 @@ class WasbHook(BaseHook):
conn_type = 'wasb'
hook_name = 'Azure Blob Storage'
+ @staticmethod
+ def get_connection_form_widgets() -> Dict[str, Any]:
+ """Returns connection widgets to add to connection form"""
+ from flask_appbuilder.fieldwidgets import BS3TextFieldWidget
+ from flask_babel import lazy_gettext
+ from wtforms import StringField
+
+ return {
+ "extra__wasb__connection_string": StringField(
+ lazy_gettext('Blob Storage Connection String (optional)'), widget=BS3TextFieldWidget()
+ ),
+ "extra__wasb__shared_access_key": StringField(
+ lazy_gettext('Blob Storage Shared Access Key (optional)'), widget=BS3TextFieldWidget()
+ ),
+ "extra__wasb__tenant_id": StringField(
+ lazy_gettext('Tenant Id (Active Directory Auth)'), widget=BS3TextFieldWidget()
+ ),
+ "extra__wasb__sas_token": StringField(
+ lazy_gettext('SAS Token (optional)'), widget=BS3TextFieldWidget()
+ ),
+ }
+
+ @staticmethod
+ def get_ui_field_behaviour() -> Dict:
+ """Returns custom field behaviour"""
+ return {
+ "hidden_fields": ['schema', 'port', 'host'],
+ "relabeling": {
+ 'login': 'Blob Storage Login (optional)',
+ 'password': 'Blob Storage Token (optional)',
Review comment:
```suggestion
'password': 'Blob Storage Key (optional)',
```
Should we use `Key` instead of `Token`? Considering that users might mistake it for SAS token?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#issuecomment-815004394
[The Workflow run](https://github.com/apache/airflow/actions/runs/726472661) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a change in pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
josh-fell commented on a change in pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#discussion_r609748699
##########
File path: airflow/providers/microsoft/azure/hooks/wasb.py
##########
@@ -57,6 +57,50 @@ class WasbHook(BaseHook):
conn_type = 'wasb'
hook_name = 'Azure Blob Storage'
+ @staticmethod
+ def get_connection_form_widgets() -> Dict[str, Any]:
+ """Returns connection widgets to add to connection form"""
+ from flask_appbuilder.fieldwidgets import BS3PasswordFieldWidget, BS3TextFieldWidget
+ from flask_babel import lazy_gettext
+ from wtforms import PasswordField, StringField
+
+ return {
+ "extra__wasb__connection_string": PasswordField(
+ lazy_gettext('Blob Storage Connection String (optional)'), widget=BS3PasswordFieldWidget()
+ ),
+ "extra__wasb__shared_access_key": PasswordField(
+ lazy_gettext('Blob Storage Shared Access Key (optional)'), widget=BS3PasswordFieldWidget()
+ ),
+ "extra__wasb__tenant_id": StringField(
+ lazy_gettext('Tenant Id (Active Directory Auth)'), widget=BS3TextFieldWidget()
+ ),
+ "extra__wasb__sas_token": PasswordField(
+ lazy_gettext('SAS Token (optional)'), widget=BS3PasswordFieldWidget()
+ ),
+ }
+
+ @staticmethod
+ def get_ui_field_behaviour() -> Dict:
+ """Returns custom field behaviour"""
+ return {
+ "hidden_fields": ['schema', 'port', 'host'],
+ "relabeling": {
+ 'login': 'Blob Storage Login (optional)',
+ 'password': 'Blob Storage Key (optional)',
+ 'host': 'Account Name (Active Directory Auth)',
+ },
+ "placeholders": {
+ 'extra': 'additional options for use with FileService and AzureFileVolume',
+ 'login': 'account name',
+ 'password': 'secret',
+ 'host': 'account url',
+ 'extra__wasb__connection_string': 'connection string auth',
+ 'extra__wasb__tenant_id': 'tenant',
Review comment:
```suggestion
'extra__wasb__tenant_id': 'tenant id',
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a change in pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
josh-fell commented on a change in pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#discussion_r609748699
##########
File path: airflow/providers/microsoft/azure/hooks/wasb.py
##########
@@ -57,6 +57,50 @@ class WasbHook(BaseHook):
conn_type = 'wasb'
hook_name = 'Azure Blob Storage'
+ @staticmethod
+ def get_connection_form_widgets() -> Dict[str, Any]:
+ """Returns connection widgets to add to connection form"""
+ from flask_appbuilder.fieldwidgets import BS3PasswordFieldWidget, BS3TextFieldWidget
+ from flask_babel import lazy_gettext
+ from wtforms import PasswordField, StringField
+
+ return {
+ "extra__wasb__connection_string": PasswordField(
+ lazy_gettext('Blob Storage Connection String (optional)'), widget=BS3PasswordFieldWidget()
+ ),
+ "extra__wasb__shared_access_key": PasswordField(
+ lazy_gettext('Blob Storage Shared Access Key (optional)'), widget=BS3PasswordFieldWidget()
+ ),
+ "extra__wasb__tenant_id": StringField(
+ lazy_gettext('Tenant Id (Active Directory Auth)'), widget=BS3TextFieldWidget()
+ ),
+ "extra__wasb__sas_token": PasswordField(
+ lazy_gettext('SAS Token (optional)'), widget=BS3PasswordFieldWidget()
+ ),
+ }
+
+ @staticmethod
+ def get_ui_field_behaviour() -> Dict:
+ """Returns custom field behaviour"""
+ return {
+ "hidden_fields": ['schema', 'port', 'host'],
+ "relabeling": {
+ 'login': 'Blob Storage Login (optional)',
+ 'password': 'Blob Storage Key (optional)',
+ 'host': 'Account Name (Active Directory Auth)',
+ },
+ "placeholders": {
+ 'extra': 'additional options for use with FileService and AzureFileVolume',
+ 'login': 'account name',
+ 'password': 'secret',
+ 'host': 'account url',
+ 'extra__wasb__connection_string': 'connection string auth',
+ 'extra__wasb__tenant_id': 'tenant',
Review comment:
```suggestion
'extra__wasb__tenant_id': 'tenant' id,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a change in pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
josh-fell commented on a change in pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#discussion_r609783462
##########
File path: airflow/providers/microsoft/azure/hooks/azure_cosmos.py
##########
@@ -50,6 +50,39 @@ class AzureCosmosDBHook(BaseHook):
conn_type = 'azure_cosmos'
hook_name = 'Azure CosmosDB'
+ @staticmethod
+ def get_connection_form_widgets() -> Dict[str, Any]:
+ """Returns connection widgets to add to connection form"""
+ from flask_appbuilder.fieldwidgets import BS3TextFieldWidget
+ from flask_babel import lazy_gettext
+ from wtforms import StringField
+
+ return {
+ "extra__azure_cosmos__database_name": StringField(
+ lazy_gettext('Cosmos Database Name'), widget=BS3TextFieldWidget()
+ ),
+ "extra__azure_cosmos__collection_name": StringField(
+ lazy_gettext('Cosmos Collection Name'), widget=BS3TextFieldWidget()
+ ),
+ }
+
+ @staticmethod
+ def get_ui_field_behaviour() -> Dict:
+ """Returns custom field behaviour"""
+ return {
+ "hidden_fields": ['schema', 'port', 'host', 'extra'],
+ "relabeling": {
+ 'login': 'Cosmos Endpoint URI',
+ 'password': 'Cosmos Master Key Token',
+ },
+ "placeholders": {
+ 'login': 'endpoint uri',
+ 'password': 'master key',
+ 'extra__azure_cosmos__database_name': 'database name (optional)',
+ 'extra__azure_cosmos__collection_name': 'collection name (optional)',
Review comment:
For consistency can the "(optional)" text in the placeholder be part of the field relabeling?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] sunkickr commented on a change in pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
sunkickr commented on a change in pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#discussion_r608926768
##########
File path: airflow/providers/microsoft/azure/hooks/wasb.py
##########
@@ -57,6 +57,50 @@ class WasbHook(BaseHook):
conn_type = 'wasb'
hook_name = 'Azure Blob Storage'
+ @staticmethod
+ def get_connection_form_widgets() -> Dict[str, Any]:
+ """Returns connection widgets to add to connection form"""
+ from flask_appbuilder.fieldwidgets import BS3TextFieldWidget
+ from flask_babel import lazy_gettext
+ from wtforms import StringField
+
+ return {
+ "extra__wasb__connection_string": StringField(
+ lazy_gettext('Blob Storage Connection String (optional)'), widget=BS3TextFieldWidget()
+ ),
+ "extra__wasb__shared_access_key": StringField(
+ lazy_gettext('Blob Storage Shared Access Key (optional)'), widget=BS3TextFieldWidget()
+ ),
+ "extra__wasb__tenant_id": StringField(
+ lazy_gettext('Tenant Id (Active Directory Auth)'), widget=BS3TextFieldWidget()
+ ),
+ "extra__wasb__sas_token": StringField(
+ lazy_gettext('SAS Token (optional)'), widget=BS3TextFieldWidget()
+ ),
+ }
+
+ @staticmethod
+ def get_ui_field_behaviour() -> Dict:
+ """Returns custom field behaviour"""
+ return {
+ "hidden_fields": ['schema', 'port', 'host'],
+ "relabeling": {
+ 'login': 'Blob Storage Login (optional)',
+ 'password': 'Blob Storage Token (optional)',
Review comment:
I agree I'll make the change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a change in pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
josh-fell commented on a change in pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#discussion_r609771391
##########
File path: airflow/providers/microsoft/azure/hooks/azure_container_instance.py
##########
@@ -41,11 +41,53 @@ class AzureContainerInstanceHook(AzureBaseHook):
:type conn_id: str
"""
- conn_name_attr = 'conn_id'
+ conn_name_attr = 'azure_conn_id'
default_conn_name = 'azure_default'
conn_type = 'azure_container_instances'
hook_name = 'Azure Container Instance'
+ @staticmethod
+ def get_connection_form_widgets() -> Dict[str, Any]:
+ """Returns connection widgets to add to connection form"""
+ from flask_appbuilder.fieldwidgets import BS3TextFieldWidget
+ from flask_babel import lazy_gettext
+ from wtforms import StringField
+
+ return {
+ "extra__azure__tenantId": StringField(
+ lazy_gettext('Azure Tenant ID'), widget=BS3TextFieldWidget()
+ ),
+ "extra__azure__subscriptionId": StringField(
+ lazy_gettext('Azure Subscription ID'), widget=BS3TextFieldWidget()
+ ),
+ }
+
+ @staticmethod
+ def get_ui_field_behaviour() -> Dict:
+ """Returns custom field behaviour"""
+ import json
+
+ return {
+ "hidden_fields": ['schema', 'port', 'host'],
+ "relabeling": {
+ 'login': 'Azure Client ID',
+ 'password': 'Azure Secret',
+ },
+ "placeholders": {
+ 'extra': json.dumps(
+ {
+ "key_path": "path to json file for auth",
+ "key_json": "specifies json dict for auth",
+ },
+ indent=1,
+ ),
+ 'login': 'client_id (token credentials auth)',
Review comment:
```suggestion
'login': 'client id (token credentials auth)',
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a change in pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
josh-fell commented on a change in pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#discussion_r609751734
##########
File path: airflow/providers/microsoft/azure/hooks/azure_data_factory.py
##########
@@ -75,13 +75,57 @@ class AzureDataFactoryHook(BaseHook): # pylint: disable=too-many-public-methods
A hook to interact with Azure Data Factory.
:param conn_id: The Azure Data Factory connection id.
+ :type conn_id: str
"""
conn_type: str = 'azure_data_factory'
conn_name_attr: str = 'azure_data_factory_conn_id'
default_conn_name: str = 'azure_data_factory_default'
hook_name: str = 'Azure Data Factory'
+ @staticmethod
+ def get_connection_form_widgets() -> Dict[str, Any]:
+ """Returns connection widgets to add to connection form"""
+ from flask_appbuilder.fieldwidgets import BS3TextFieldWidget
+ from flask_babel import lazy_gettext
+ from wtforms import StringField
+
+ return {
+ "extra__azure_data_factory__tenantId": StringField(
+ lazy_gettext('Azure Tenant ID'), widget=BS3TextFieldWidget()
+ ),
+ "extra__azure_data_factory__subscriptionId": StringField(
+ lazy_gettext('Azure Subscription ID'), widget=BS3TextFieldWidget()
+ ),
+ }
+
+ @staticmethod
+ def get_ui_field_behaviour() -> Dict:
+ """Returns custom field behaviour"""
+ import json
+
+ return {
+ "hidden_fields": ['schema', 'port', 'host'],
+ "relabeling": {
+ 'login': 'Azure Client ID',
+ 'password': 'Azure Secret',
+ 'extra': 'Extra (optional)',
+ },
+ "placeholders": {
+ 'extra': json.dumps(
+ {
+ "resourceGroup": "azure resource group name",
+ "factory": "azure data factory name",
+ },
+ indent=1,
+ ),
+ 'login': 'client_id',
Review comment:
Making consistent with other instances of IDs in placeholders
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a change in pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
josh-fell commented on a change in pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#discussion_r609747789
##########
File path: airflow/providers/microsoft/azure/hooks/azure_data_lake.py
##########
@@ -49,6 +49,39 @@ class AzureDataLakeHook(BaseHook):
conn_type = 'azure_data_lake'
hook_name = 'Azure Data Lake'
+ @staticmethod
+ def get_connection_form_widgets() -> Dict[str, Any]:
+ """Returns connection widgets to add to connection form"""
+ from flask_appbuilder.fieldwidgets import BS3TextFieldWidget
+ from flask_babel import lazy_gettext
+ from wtforms import StringField
+
+ return {
+ "extra__azure_data_lake__tenant": StringField(
+ lazy_gettext('Azure Tenant ID'), widget=BS3TextFieldWidget()
+ ),
+ "extra__azure_data_lake__account_name": StringField(
+ lazy_gettext('Azure DataLake Store Name'), widget=BS3TextFieldWidget()
+ ),
+ }
+
+ @staticmethod
+ def get_ui_field_behaviour() -> Dict:
+ """Returns custom field behaviour"""
+ return {
+ "hidden_fields": ['schema', 'port', 'host', 'extra'],
+ "relabeling": {
+ 'login': 'Azure Client ID',
+ 'password': 'Azure Client Secret',
+ },
+ "placeholders": {
+ 'login': 'client_id',
+ 'password': 'secret',
+ 'extra__azure_data_lake__tenant': 'tenent id',
Review comment:
```suggestion
'extra__azure_data_lake__tenant': 'tenant id',
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a change in pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
josh-fell commented on a change in pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#discussion_r609772159
##########
File path: airflow/providers/microsoft/azure/hooks/azure_container_instance.py
##########
@@ -41,11 +41,53 @@ class AzureContainerInstanceHook(AzureBaseHook):
:type conn_id: str
"""
- conn_name_attr = 'conn_id'
+ conn_name_attr = 'azure_conn_id'
default_conn_name = 'azure_default'
conn_type = 'azure_container_instances'
hook_name = 'Azure Container Instance'
+ @staticmethod
+ def get_connection_form_widgets() -> Dict[str, Any]:
+ """Returns connection widgets to add to connection form"""
+ from flask_appbuilder.fieldwidgets import BS3TextFieldWidget
+ from flask_babel import lazy_gettext
+ from wtforms import StringField
+
+ return {
+ "extra__azure__tenantId": StringField(
+ lazy_gettext('Azure Tenant ID'), widget=BS3TextFieldWidget()
+ ),
+ "extra__azure__subscriptionId": StringField(
+ lazy_gettext('Azure Subscription ID'), widget=BS3TextFieldWidget()
+ ),
+ }
+
+ @staticmethod
+ def get_ui_field_behaviour() -> Dict:
+ """Returns custom field behaviour"""
+ import json
+
+ return {
+ "hidden_fields": ['schema', 'port', 'host'],
+ "relabeling": {
+ 'login': 'Azure Client ID',
+ 'password': 'Azure Secret',
+ },
+ "placeholders": {
+ 'extra': json.dumps(
+ {
+ "key_path": "path to json file for auth",
+ "key_json": "specifies json dict for auth",
+ },
+ indent=1,
+ ),
+ 'login': 'client_id (token credentials auth)',
+ 'password': 'secret (token credentials auth)',
+ 'extra__azure__tenantId': 'tenentId (token credentials auth)',
+ 'extra__azure__subscriptionId': 'subscriptionId (token credentials auth)',
Review comment:
```suggestion
'extra__azure__subscriptionId': 'subscription Id (token credentials auth)',
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a change in pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
josh-fell commented on a change in pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#discussion_r609783462
##########
File path: airflow/providers/microsoft/azure/hooks/azure_cosmos.py
##########
@@ -50,6 +50,39 @@ class AzureCosmosDBHook(BaseHook):
conn_type = 'azure_cosmos'
hook_name = 'Azure CosmosDB'
+ @staticmethod
+ def get_connection_form_widgets() -> Dict[str, Any]:
+ """Returns connection widgets to add to connection form"""
+ from flask_appbuilder.fieldwidgets import BS3TextFieldWidget
+ from flask_babel import lazy_gettext
+ from wtforms import StringField
+
+ return {
+ "extra__azure_cosmos__database_name": StringField(
+ lazy_gettext('Cosmos Database Name'), widget=BS3TextFieldWidget()
+ ),
+ "extra__azure_cosmos__collection_name": StringField(
+ lazy_gettext('Cosmos Collection Name'), widget=BS3TextFieldWidget()
+ ),
+ }
+
+ @staticmethod
+ def get_ui_field_behaviour() -> Dict:
+ """Returns custom field behaviour"""
+ return {
+ "hidden_fields": ['schema', 'port', 'host', 'extra'],
+ "relabeling": {
+ 'login': 'Cosmos Endpoint URI',
+ 'password': 'Cosmos Master Key Token',
+ },
+ "placeholders": {
+ 'login': 'endpoint uri',
+ 'password': 'master key',
+ 'extra__azure_cosmos__database_name': 'database name (optional)',
+ 'extra__azure_cosmos__collection_name': 'collection name (optional)',
Review comment:
For consistency can the "(optional)" text in the placeholders for Cosmos Database Name and Cosmos Collection Name be part of the field relabeling?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] sunkickr commented on pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
sunkickr commented on pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#issuecomment-816966087
Thanks @josh-fell I'll implement these changes now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#issuecomment-814467725
[The Workflow run](https://github.com/apache/airflow/actions/runs/724006230) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] sunkickr commented on a change in pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
sunkickr commented on a change in pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#discussion_r606263524
##########
File path: airflow/providers/microsoft/azure/hooks/base_azure.py
##########
@@ -30,14 +30,56 @@ class AzureBaseHook(BaseHook):
authenticate the client library used for upstream azure hooks.
:param sdk_client: The SDKClient to use.
+ :type sdk_client: Optional[str]
:param conn_id: The azure connection id which refers to the information to connect to the service.
+ :type conn_id: str
"""
- conn_name_attr = 'conn_id'
+ conn_name_attr = 'azure_conn_id'
Review comment:
wondering what implication this change has
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a change in pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
josh-fell commented on a change in pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#discussion_r609750950
##########
File path: airflow/providers/microsoft/azure/hooks/azure_data_factory.py
##########
@@ -75,13 +75,57 @@ class AzureDataFactoryHook(BaseHook): # pylint: disable=too-many-public-methods
A hook to interact with Azure Data Factory.
:param conn_id: The Azure Data Factory connection id.
+ :type conn_id: str
"""
conn_type: str = 'azure_data_factory'
conn_name_attr: str = 'azure_data_factory_conn_id'
default_conn_name: str = 'azure_data_factory_default'
hook_name: str = 'Azure Data Factory'
+ @staticmethod
+ def get_connection_form_widgets() -> Dict[str, Any]:
+ """Returns connection widgets to add to connection form"""
+ from flask_appbuilder.fieldwidgets import BS3TextFieldWidget
+ from flask_babel import lazy_gettext
+ from wtforms import StringField
+
+ return {
+ "extra__azure_data_factory__tenantId": StringField(
+ lazy_gettext('Azure Tenant ID'), widget=BS3TextFieldWidget()
+ ),
+ "extra__azure_data_factory__subscriptionId": StringField(
+ lazy_gettext('Azure Subscription ID'), widget=BS3TextFieldWidget()
+ ),
+ }
+
+ @staticmethod
+ def get_ui_field_behaviour() -> Dict:
+ """Returns custom field behaviour"""
+ import json
+
+ return {
+ "hidden_fields": ['schema', 'port', 'host'],
+ "relabeling": {
+ 'login': 'Azure Client ID',
+ 'password': 'Azure Secret',
+ 'extra': 'Extra (optional)',
+ },
+ "placeholders": {
+ 'extra': json.dumps(
+ {
+ "resourceGroup": "azure resource group name",
+ "factory": "azure data factory name",
+ },
+ indent=1,
+ ),
+ 'login': 'client_id',
Review comment:
```suggestion
'login': 'client id',
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a change in pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
josh-fell commented on a change in pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#discussion_r609747014
##########
File path: airflow/providers/microsoft/azure/hooks/azure_container_instance.py
##########
@@ -41,11 +41,53 @@ class AzureContainerInstanceHook(AzureBaseHook):
:type conn_id: str
"""
- conn_name_attr = 'conn_id'
+ conn_name_attr = 'azure_conn_id'
default_conn_name = 'azure_default'
conn_type = 'azure_container_instances'
hook_name = 'Azure Container Instance'
+ @staticmethod
+ def get_connection_form_widgets() -> Dict[str, Any]:
+ """Returns connection widgets to add to connection form"""
+ from flask_appbuilder.fieldwidgets import BS3TextFieldWidget
+ from flask_babel import lazy_gettext
+ from wtforms import StringField
+
+ return {
+ "extra__azure__tenantId": StringField(
+ lazy_gettext('Azure Tenant ID'), widget=BS3TextFieldWidget()
+ ),
+ "extra__azure__subscriptionId": StringField(
+ lazy_gettext('Azure Subscription ID'), widget=BS3TextFieldWidget()
+ ),
+ }
+
+ @staticmethod
+ def get_ui_field_behaviour() -> Dict:
+ """Returns custom field behaviour"""
+ import json
+
+ return {
+ "hidden_fields": ['schema', 'port', 'host'],
+ "relabeling": {
+ 'login': 'Azure Client ID',
+ 'password': 'Azure Secret',
+ },
+ "placeholders": {
+ 'extra': json.dumps(
+ {
+ "key_path": "path to json file for auth",
+ "key_json": "specifies json dict for auth",
+ },
+ indent=1,
+ ),
+ 'login': 'client_id (token credentials auth)',
+ 'password': 'secret (token credentials auth)',
+ 'extra__azure__tenantId': 'tenentId (token credentials auth)',
Review comment:
```suggestion
'extra__azure__tenantId': 'tenant Id (token credentials auth)',
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#issuecomment-815128992
[The Workflow run](https://github.com/apache/airflow/actions/runs/726889926) is cancelling this PR. Building images for the PR has failed. Follow the workflow link to check the reason.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#issuecomment-814483838
[The Workflow run](https://github.com/apache/airflow/actions/runs/724138432) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#issuecomment-818291028
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest master at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a change in pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
josh-fell commented on a change in pull request #15159:
URL: https://github.com/apache/airflow/pull/15159#discussion_r609749045
##########
File path: airflow/providers/microsoft/azure/hooks/wasb.py
##########
@@ -57,6 +57,50 @@ class WasbHook(BaseHook):
conn_type = 'wasb'
hook_name = 'Azure Blob Storage'
+ @staticmethod
+ def get_connection_form_widgets() -> Dict[str, Any]:
+ """Returns connection widgets to add to connection form"""
+ from flask_appbuilder.fieldwidgets import BS3PasswordFieldWidget, BS3TextFieldWidget
+ from flask_babel import lazy_gettext
+ from wtforms import PasswordField, StringField
+
+ return {
+ "extra__wasb__connection_string": PasswordField(
+ lazy_gettext('Blob Storage Connection String (optional)'), widget=BS3PasswordFieldWidget()
+ ),
+ "extra__wasb__shared_access_key": PasswordField(
+ lazy_gettext('Blob Storage Shared Access Key (optional)'), widget=BS3PasswordFieldWidget()
+ ),
+ "extra__wasb__tenant_id": StringField(
+ lazy_gettext('Tenant Id (Active Directory Auth)'), widget=BS3TextFieldWidget()
+ ),
+ "extra__wasb__sas_token": PasswordField(
+ lazy_gettext('SAS Token (optional)'), widget=BS3PasswordFieldWidget()
+ ),
+ }
+
+ @staticmethod
+ def get_ui_field_behaviour() -> Dict:
+ """Returns custom field behaviour"""
+ return {
+ "hidden_fields": ['schema', 'port', 'host'],
+ "relabeling": {
+ 'login': 'Blob Storage Login (optional)',
+ 'password': 'Blob Storage Key (optional)',
+ 'host': 'Account Name (Active Directory Auth)',
+ },
+ "placeholders": {
+ 'extra': 'additional options for use with FileService and AzureFileVolume',
+ 'login': 'account name',
+ 'password': 'secret',
+ 'host': 'account url',
+ 'extra__wasb__connection_string': 'connection string auth',
+ 'extra__wasb__tenant_id': 'tenant',
Review comment:
Making this consistent with other instances of Tenant ID
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil merged pull request #15159: Add dynamic connection fields to Azure Connection
Posted by GitBox <gi...@apache.org>.
kaxil merged pull request #15159:
URL: https://github.com/apache/airflow/pull/15159
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org