You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu> on 2021/09/09 08:45:31 UTC

Change in asterixdb[master]: [WIP]: Adding support to managedIdentityId

From Hussain Towaileb <hu...@gmail.com>:

Hussain Towaileb has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13143 )


Change subject: [WIP]: Adding support to managedIdentityId
......................................................................

[WIP]: Adding support to managedIdentityId

Change-Id: I9eb01a7fb53197ad47aab7b87f7d8d48c21782db
---
M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
2 files changed, 17 insertions(+), 0 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/43/13143/1

diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
index 243b87c..0b4811d 100644
--- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
+++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
@@ -356,6 +356,8 @@
         public static final String CLIENT_SECRET_FIELD_NAME = "clientSecret";
         public static final String CLIENT_CERTIFICATE_FIELD_NAME = "clientCertificate";
         public static final String CLIENT_CERTIFICATE_PASSWORD_FIELD_NAME = "clientCertificatePassword";
+        public static final String MANAGED_IDENTITY_ID_FIELD_NAME = "managedIdentityId";
+        public static final String ENDPOINT_FIELD_NAME = "endpoint";
     }
 
     public static class GCS {
diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
index dd44436..bbaab12 100644
--- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
+++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
@@ -40,6 +40,8 @@
 import static org.apache.asterix.external.util.ExternalDataConstants.AzureBlob.CLIENT_ID_FIELD_NAME;
 import static org.apache.asterix.external.util.ExternalDataConstants.AzureBlob.CLIENT_SECRET_FIELD_NAME;
 import static org.apache.asterix.external.util.ExternalDataConstants.AzureBlob.CONNECTION_STRING_FIELD_NAME;
+import static org.apache.asterix.external.util.ExternalDataConstants.AzureBlob.ENDPOINT_FIELD_NAME;
+import static org.apache.asterix.external.util.ExternalDataConstants.AzureBlob.MANAGED_IDENTITY_ID_FIELD_NAME;
 import static org.apache.asterix.external.util.ExternalDataConstants.AzureBlob.TENANT_ID_FIELD_NAME;
 import static org.apache.asterix.external.util.ExternalDataConstants.GCS.JSON_CREDENTIALS_FIELD_NAME;
 import static org.apache.asterix.external.util.ExternalDataConstants.KEY_ADAPTER_NAME_GCS;
@@ -115,6 +117,7 @@
 
 import com.azure.identity.ClientCertificateCredentialBuilder;
 import com.azure.identity.ClientSecretCredentialBuilder;
+import com.azure.identity.ManagedIdentityCredentialBuilder;
 import com.azure.storage.blob.BlobContainerClient;
 import com.azure.storage.blob.BlobServiceClient;
 import com.azure.storage.blob.BlobServiceClientBuilder;
@@ -1227,11 +1230,13 @@
         public static BlobServiceClient buildAzureClient(Map<String, String> configuration)
                 throws CompilationException {
             String connectionString = configuration.get(CONNECTION_STRING_FIELD_NAME);
+            String managedIdentityId = configuration.get(MANAGED_IDENTITY_ID_FIELD_NAME);
             String tenantId = configuration.get(TENANT_ID_FIELD_NAME);
             String clientId = configuration.get(CLIENT_ID_FIELD_NAME);
             String clientSecret = configuration.get(CLIENT_SECRET_FIELD_NAME);
             String clientCertificate = configuration.get(CLIENT_CERTIFICATE_FIELD_NAME);
             String clientCertificatePassword = configuration.get(CLIENT_CERTIFICATE_PASSWORD_FIELD_NAME);
+            String endpoint = configuration.get(ENDPOINT_FIELD_NAME);
 
             // Client builder
             BlobServiceClientBuilder builder = new BlobServiceClientBuilder();
@@ -1245,6 +1250,16 @@
                 }
             }
 
+            // Endpoint is required
+            if (endpoint != null) {
+                builder.endpoint(endpoint);
+            }
+
+            // Managed identity credentials
+            if (managedIdentityId != null) {
+                builder.credential(new ManagedIdentityCredentialBuilder().clientId(managedIdentityId).build());
+            }
+
             // Active Directory authentication
             if (clientId != null) {
                 // Both (or neither) client secret and client secret were provided, only one is allowed

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13143
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I9eb01a7fb53197ad47aab7b87f7d8d48c21782db
Gerrit-Change-Number: 13143
Gerrit-PatchSet: 1
Gerrit-Owner: Hussain Towaileb <hu...@gmail.com>
Gerrit-MessageType: newchange

Change in asterixdb[master]: [WIP]: Adding support to managedIdentityId

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13143 )

Change subject: [WIP]: Adding support to managedIdentityId
......................................................................


Patch Set 3:

Analytics Compatibility Compilation Successful
https://cbjenkins.page.link/RJqc3uQf6B2A6DH9A : SUCCESS


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13143
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I9eb01a7fb53197ad47aab7b87f7d8d48c21782db
Gerrit-Change-Number: 13143
Gerrit-PatchSet: 3
Gerrit-Owner: Hussain Towaileb <hu...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Sun, 12 Sep 2021 19:21:33 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: No
Gerrit-MessageType: comment

Change in asterixdb[master]: [WIP]: Adding support to managedIdentityId

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Hussain Towaileb <hu...@gmail.com>:

Hussain Towaileb has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13143 )


Change subject: [WIP]: Adding support to managedIdentityId
......................................................................

[WIP]: Adding support to managedIdentityId

Change-Id: I9eb01a7fb53197ad47aab7b87f7d8d48c21782db
---
M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
2 files changed, 17 insertions(+), 0 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/43/13143/1

diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
index 243b87c..0b4811d 100644
--- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
+++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
@@ -356,6 +356,8 @@
         public static final String CLIENT_SECRET_FIELD_NAME = "clientSecret";
         public static final String CLIENT_CERTIFICATE_FIELD_NAME = "clientCertificate";
         public static final String CLIENT_CERTIFICATE_PASSWORD_FIELD_NAME = "clientCertificatePassword";
+        public static final String MANAGED_IDENTITY_ID_FIELD_NAME = "managedIdentityId";
+        public static final String ENDPOINT_FIELD_NAME = "endpoint";
     }
 
     public static class GCS {
diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
index dd44436..bbaab12 100644
--- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
+++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
@@ -40,6 +40,8 @@
 import static org.apache.asterix.external.util.ExternalDataConstants.AzureBlob.CLIENT_ID_FIELD_NAME;
 import static org.apache.asterix.external.util.ExternalDataConstants.AzureBlob.CLIENT_SECRET_FIELD_NAME;
 import static org.apache.asterix.external.util.ExternalDataConstants.AzureBlob.CONNECTION_STRING_FIELD_NAME;
+import static org.apache.asterix.external.util.ExternalDataConstants.AzureBlob.ENDPOINT_FIELD_NAME;
+import static org.apache.asterix.external.util.ExternalDataConstants.AzureBlob.MANAGED_IDENTITY_ID_FIELD_NAME;
 import static org.apache.asterix.external.util.ExternalDataConstants.AzureBlob.TENANT_ID_FIELD_NAME;
 import static org.apache.asterix.external.util.ExternalDataConstants.GCS.JSON_CREDENTIALS_FIELD_NAME;
 import static org.apache.asterix.external.util.ExternalDataConstants.KEY_ADAPTER_NAME_GCS;
@@ -115,6 +117,7 @@
 
 import com.azure.identity.ClientCertificateCredentialBuilder;
 import com.azure.identity.ClientSecretCredentialBuilder;
+import com.azure.identity.ManagedIdentityCredentialBuilder;
 import com.azure.storage.blob.BlobContainerClient;
 import com.azure.storage.blob.BlobServiceClient;
 import com.azure.storage.blob.BlobServiceClientBuilder;
@@ -1227,11 +1230,13 @@
         public static BlobServiceClient buildAzureClient(Map<String, String> configuration)
                 throws CompilationException {
             String connectionString = configuration.get(CONNECTION_STRING_FIELD_NAME);
+            String managedIdentityId = configuration.get(MANAGED_IDENTITY_ID_FIELD_NAME);
             String tenantId = configuration.get(TENANT_ID_FIELD_NAME);
             String clientId = configuration.get(CLIENT_ID_FIELD_NAME);
             String clientSecret = configuration.get(CLIENT_SECRET_FIELD_NAME);
             String clientCertificate = configuration.get(CLIENT_CERTIFICATE_FIELD_NAME);
             String clientCertificatePassword = configuration.get(CLIENT_CERTIFICATE_PASSWORD_FIELD_NAME);
+            String endpoint = configuration.get(ENDPOINT_FIELD_NAME);
 
             // Client builder
             BlobServiceClientBuilder builder = new BlobServiceClientBuilder();
@@ -1245,6 +1250,16 @@
                 }
             }
 
+            // Endpoint is required
+            if (endpoint != null) {
+                builder.endpoint(endpoint);
+            }
+
+            // Managed identity credentials
+            if (managedIdentityId != null) {
+                builder.credential(new ManagedIdentityCredentialBuilder().clientId(managedIdentityId).build());
+            }
+
             // Active Directory authentication
             if (clientId != null) {
                 // Both (or neither) client secret and client secret were provided, only one is allowed

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13143
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I9eb01a7fb53197ad47aab7b87f7d8d48c21782db
Gerrit-Change-Number: 13143
Gerrit-PatchSet: 1
Gerrit-Owner: Hussain Towaileb <hu...@gmail.com>
Gerrit-MessageType: newchange

Change in asterixdb[master]: [WIP]: Adding support to managedIdentityId

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Jenkins <je...@fulliautomatix.ics.uci.edu>:

Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13143 )

Change subject: [WIP]: Adding support to managedIdentityId
......................................................................


Patch Set 1: Integration-Tests-1

Integration Tests Failed

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/12454/ : FAILURE


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13143
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I9eb01a7fb53197ad47aab7b87f7d8d48c21782db
Gerrit-Change-Number: 13143
Gerrit-PatchSet: 1
Gerrit-Owner: Hussain Towaileb <hu...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-CC: Anon. E. Moose #1000171
Gerrit-Comment-Date: Thu, 09 Sep 2021 09:39:19 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment

Change in asterixdb[master]: [ASTERIXDB-2935]: Add support to managedIdentityId

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Jenkins <je...@fulliautomatix.ics.uci.edu>:

Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13143 )

Change subject: [ASTERIXDB-2935]: Add support to managedIdentityId
......................................................................


Patch Set 4: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/12498/ : SUCCESS


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13143
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I9eb01a7fb53197ad47aab7b87f7d8d48c21782db
Gerrit-Change-Number: 13143
Gerrit-PatchSet: 4
Gerrit-Owner: Hussain Towaileb <hu...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Thu, 16 Sep 2021 12:50:45 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment

Change in asterixdb[master]: [ASTERIXDB-2935]: Add support to managedIdentityId

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13143 )

Change subject: [ASTERIXDB-2935]: Add support to managedIdentityId
......................................................................


Patch Set 5: Contrib-2

Analytics Compatibility Tests Failed
https://cbjenkins.page.link/KQHmF7fGRhTsB8Pv6 : UNSTABLE


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13143
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I9eb01a7fb53197ad47aab7b87f7d8d48c21782db
Gerrit-Change-Number: 13143
Gerrit-PatchSet: 5
Gerrit-Owner: Hussain Towaileb <hu...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Sun, 19 Sep 2021 21:31:29 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment

Change in asterixdb[master]: [ASTERIXDB-2935]: Add support to managedIdentityId

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Wael Alkowaileet <wa...@gmail.com>:

Wael Alkowaileet has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13143 )

Change subject: [ASTERIXDB-2935]: Add support to managedIdentityId
......................................................................


Patch Set 5: Code-Review+2


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13143
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I9eb01a7fb53197ad47aab7b87f7d8d48c21782db
Gerrit-Change-Number: 13143
Gerrit-PatchSet: 5
Gerrit-Owner: Hussain Towaileb <hu...@gmail.com>
Gerrit-Reviewer: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Hussain Towaileb <hu...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Michael Blow <mb...@apache.org>
Gerrit-Reviewer: Murtadha Hubail <mh...@apache.org>
Gerrit-Reviewer: Till Westmann <ti...@apache.org>
Gerrit-Reviewer: Wael Alkowaileet <wa...@gmail.com>
Gerrit-Comment-Date: Mon, 20 Sep 2021 14:57:16 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment

Change in asterixdb[master]: [ASTERIXDB-2935]: Add support to managedIdentityId

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13143 )

Change subject: [ASTERIXDB-2935]: Add support to managedIdentityId
......................................................................


Patch Set 4:

Analytics Compatibility Compilation Successful
https://cbjenkins.page.link/BZRCPKWh6JVcKQHx9 : SUCCESS


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13143
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I9eb01a7fb53197ad47aab7b87f7d8d48c21782db
Gerrit-Change-Number: 13143
Gerrit-PatchSet: 4
Gerrit-Owner: Hussain Towaileb <hu...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Thu, 16 Sep 2021 07:50:23 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: No
Gerrit-MessageType: comment

Change in asterixdb[master]: [ASTERIXDB-2935]: Add support to managedIdentityId

Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13143 )

Change subject: [ASTERIXDB-2935]: Add support to managedIdentityId
......................................................................


Patch Set 4: Contrib-2

Analytics Compatibility Tests Failed
https://cbjenkins.page.link/eZJoHb3BZv2M1GuG9 : UNSTABLE


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13143
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I9eb01a7fb53197ad47aab7b87f7d8d48c21782db
Gerrit-Change-Number: 13143
Gerrit-PatchSet: 4
Gerrit-Owner: Hussain Towaileb <hu...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Thu, 16 Sep 2021 14:35:17 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment