[airflow] branch master updated: Fixes problems with extras for custom connection types (#13640)

[po...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/master by this push:
     new b007fc3  Fixes problems with extras for custom connection types (#13640)
b007fc3 is described below

commit b007fc33d481f0f1341d1e1e4cba719a5fe6580d
Author: Jarek Potiuk <ja...@polidea.com>
AuthorDate: Wed Jan 13 00:32:49 2021 +0100

    Fixes problems with extras for custom connection types (#13640)
    
    The custom providers with custom connections can define
    extra widgets and fields, however there were problems with
    those custom fields in Aiflow 2.0.0:
    
    * When connection type was a subset of another connection
      type (for example jdbc and jdbcx) widgets from the
      'subset' connection type appeared also in the 'superset'
      one due to prefix matching in javascript.
    
    * Each connection when saved received 'full set' of extra
      fields from other connection types (with empty values).
      This problem is likely present in Airflow 1.10 but due
      to limited number of connections supported it had no
      real implications besides slightly bigger dictionary
      stored in 'extra' field.
    
    * The extra field values were not saved for custom connections.
      Only the predefined connection types could save extras in
      extras field.
    
    This PR fixes it by:
    
    * adding __ matching for javascript to match only full connection
      types not prefixes
    * saving only the fields matching extra__<conn_type> when the
      connection is saved
    * removing filtering on 'known' connection types (the above
      filtering on `extra__` results in empty extra for
      connections that do not have any extra field defined.
    
    Fixes #13597
---
 airflow/www/static/js/connection_form.js |  2 +-
 airflow/www/views.py                     | 10 +++++++---
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/airflow/www/static/js/connection_form.js b/airflow/www/static/js/connection_form.js
index 6bfa11a..af6601b 100644
--- a/airflow/www/static/js/connection_form.js
+++ b/airflow/www/static/js/connection_form.js
@@ -31,7 +31,7 @@ $(document).ready(function () {
     $.each($("[id^='extra__']"), function () {
       $(this).parent().parent().addClass('hide')
     });
-    $.each($("[id^='extra__" + connectionType + "']"), function () {
+    $.each($("[id^='extra__" + connectionType + "__']"), function () {
       $(this).parent().parent().removeClass('hide')
     });
     $("label[orig_text]").each(function () {
diff --git a/airflow/www/views.py b/airflow/www/views.py
index 3bfa7c4..1d9a8ba 100644
--- a/airflow/www/views.py
+++ b/airflow/www/views.py
@@ -2894,9 +2894,13 @@ class ConnectionModelView(AirflowModelView):
 
     def process_form(self, form, is_created):
         """Process form data."""
-        formdata = form.data
-        if formdata['conn_type'] in ['jdbc', 'google_cloud_platform', 'grpc', 'yandexcloud', 'kubernetes']:
-            extra = {key: formdata[key] for key in self.extra_fields if key in formdata}
+        conn_type = form.data['conn_type']
+        extra = {
+            key: form.data[key]
+            for key in self.extra_fields
+            if key in form.data and key.startswith(f"extra__{conn_type}__")
+        }
+        if extra.keys():
             form.extra.data = json.dumps(extra)
 
     def prefill_form(self, form, pk):