You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by "kumaab (via GitHub)" <gi...@apache.org> on 2026/03/27 02:15:56 UTC
[PR] RANGER-5522: update tagsync to process Atlas notifications for Trino entities [ranger]
kumaab opened a new pull request, #895:
URL: https://github.com/apache/ranger/pull/895
## What changes were proposed in this pull request?
Atlas added support for Trino entity-types via [ATLAS-5021](https://issues.apache.org/jira/browse/ATLAS-5021). To enable use of tag-based access and masking policies in Ranger plugin for Trino, Ranger tagsync should be updated to process Atlas notifications for following Trino entity types:
```
trino_instance
trino_catalog
trino_schema
trino_table
trino_column
```
## How was this patch tested?
- successful build
- tested with CLI in `kafka` container using below configs:
```
kinit -kt /etc/keytabs/kafka.keytab kafka/ranger-kafka.rangernw@EXAMPLE.COM
cat >/tmp/kafka-client-jaas.conf <<'EOF'
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
storeKey=true
serviceName="kafka"
keyTab="/etc/keytabs/kafka.keytab"
principal="kafka/ranger-kafka.rangernw@EXAMPLE.COM";
};
EOF
cat >/tmp/kafka-client.properties <<'EOF'
security.protocol=SASL_PLAINTEXT
sasl.mechanism=GSSAPI
sasl.kerberos.service.name=kafka
EOF
export KAFKA_OPTS="-Djava.security.krb5.conf=/etc/krb5.conf -Djava.security.auth.login.config=/tmp/kafka-client-jaas.conf"
cat <<'EOF' | /opt/kafka/bin/kafka-console-producer.sh --bootstrap-server ranger-kafka.rangernw:9092 --topic ATLAS_ENTITIES --producer.config /tmp/kafka-client.properties {"version":{"version":"1.0.0"},"msgCreationTime":1710000002000,"spooled":false,"message":{"type":"ENTITY_NOTIFICATION_V2","operationType":"CLASSIFICATION_ADD","eventTime":1710000002000,"entity":{"typeName":"trino_column","guid":"32222222-2222-2222-2222-222222222224","status":"ACTIVE","displayText":"customer_id","isIncomplete":false,"attributes":{"qualifiedName":"sales.reporting.orders.customer_id@dev"},"classificationNames":["SENSITIVE"],"classifications":[{"typeName":"SENSITIVE","entityGuid":"32222222-2222-2222-2222-222222222224","entityStatus":"ACTIVE","propagate":false,"removePropagationsOnEntityDelete":false,"attributes":{"kind":"customer"}}]}}}
EOF
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] RANGER-5522: update tagsync to process Atlas notifications for Trino entities [ranger]
Posted by "mneethiraj (via GitHub)" <gi...@apache.org>.
mneethiraj commented on code in PR #895:
URL: https://github.com/apache/ranger/pull/895#discussion_r3003801777
##########
tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasTrinoResourceMapper.java:
##########
@@ -0,0 +1,123 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ranger.tagsync.source.atlas;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyResource;
+import org.apache.ranger.plugin.model.RangerServiceResource;
+import org.apache.ranger.tagsync.source.atlasrest.RangerAtlasEntity;
+
+import java.util.HashMap;
+import java.util.Map;
+
+public class AtlasTrinoResourceMapper extends AtlasResourceMapper {
+ public static final String ENTITY_TYPE_TRINO_CATALOG = "trino_catalog";
+ public static final String ENTITY_TYPE_TRINO_SCHEMA = "trino_schema";
+ public static final String ENTITY_TYPE_TRINO_TABLE = "trino_table";
+ public static final String ENTITY_TYPE_TRINO_COLUMN = "trino_column";
+ public static final String RANGER_TYPE_TRINO_CATALOG = "catalog";
+ public static final String RANGER_TYPE_TRINO_SCHEMA = "schema";
+ public static final String RANGER_TYPE_TRINO_TABLE = "table";
+ public static final String RANGER_TYPE_TRINO_COLUMN = "column";
+
+ public static final String[] SUPPORTED_ENTITY_TYPES = {
+ ENTITY_TYPE_TRINO_CATALOG,
+ ENTITY_TYPE_TRINO_SCHEMA,
+ ENTITY_TYPE_TRINO_TABLE,
+ ENTITY_TYPE_TRINO_COLUMN
+ };
+
+ public AtlasTrinoResourceMapper() {
+ super("trino", SUPPORTED_ENTITY_TYPES);
+ }
+
+ /*
+ * qualifiedName can be of format, depending upon the entity-type:
+ * trino_instance: <instanceName>
Review Comment:
Remove line 53.
##########
tagsync/conf.dist/logback.xml:
##########
@@ -31,6 +31,9 @@
<cleanHistoryOnStart>true</cleanHistoryOnStart>
</rollingPolicy>
</appender>
+ <logger name="org.apache.ranger" additivity="false" level="info"> <!-- toggle to debug for testing -->
Review Comment:
Revert updates in this file as they are not related to this enhancement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] RANGER-5522: update tagsync to process Atlas notifications for Trino entities [ranger]
Posted by "mneethiraj (via GitHub)" <gi...@apache.org>.
mneethiraj commented on code in PR #895:
URL: https://github.com/apache/ranger/pull/895#discussion_r3002636498
##########
tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasTrinoResourceMapper.java:
##########
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ranger.tagsync.source.atlas;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyResource;
+import org.apache.ranger.plugin.model.RangerServiceResource;
+import org.apache.ranger.tagsync.source.atlasrest.RangerAtlasEntity;
+
+import java.util.HashMap;
+import java.util.Map;
+
+public class AtlasTrinoResourceMapper extends AtlasResourceMapper {
+ public static final String ENTITY_TYPE_TRINO_INSTANCE = "trino_instance";
+ public static final String ENTITY_TYPE_TRINO_CATALOG = "trino_catalog";
+ public static final String ENTITY_TYPE_TRINO_SCHEMA = "trino_schema";
+ public static final String ENTITY_TYPE_TRINO_TABLE = "trino_table";
+ public static final String ENTITY_TYPE_TRINO_COLUMN = "trino_column";
+ public static final String RANGER_TYPE_TRINO_CATALOG = "catalog";
+ public static final String RANGER_TYPE_TRINO_SCHEMA = "schema";
+ public static final String RANGER_TYPE_TRINO_TABLE = "table";
+ public static final String RANGER_TYPE_TRINO_COLUMN = "column";
+
+ public static final String[] SUPPORTED_ENTITY_TYPES = {
+ ENTITY_TYPE_TRINO_INSTANCE,
+ ENTITY_TYPE_TRINO_CATALOG,
+ ENTITY_TYPE_TRINO_SCHEMA,
+ ENTITY_TYPE_TRINO_TABLE,
+ ENTITY_TYPE_TRINO_COLUMN
+ };
+
+ public AtlasTrinoResourceMapper() {
+ super("trino", SUPPORTED_ENTITY_TYPES);
+ }
+
+ /*
+ * qualifiedName can be of format, depending upon the entity-type:
+ * trino_instance: <instanceName>
+ * trino_catalog: <catalog>@<instanceName>
+ * trino_schema: <catalog>.<schema>@<instanceName>
+ * trino_table: <catalog>.<schema>.<table>@<instanceName>
+ * trino_column: <catalog>.<schema>.<table>.<column>@<instanceName>
+ */
+ @Override
+ public RangerServiceResource buildResource(RangerAtlasEntity entity) throws Exception {
+ String qualifiedName = (String) entity.getAttributes().get(AtlasResourceMapper.ENTITY_ATTRIBUTE_QUALIFIED_NAME);
+
+ if (StringUtils.isEmpty(qualifiedName)) {
+ throw new Exception("attribute '" + ENTITY_ATTRIBUTE_QUALIFIED_NAME + "' not found in entity");
+ }
+
+ String entityType = entity.getTypeName();
+ String entityGuid = entity.getGuid();
+ String resourceStr = getResourceNameFromQualifiedName(qualifiedName);
+
+ if (StringUtils.isEmpty(resourceStr)) {
+ throwExceptionWithMessage("resource not found in attribute '" + ENTITY_ATTRIBUTE_QUALIFIED_NAME + "': " + qualifiedName);
+ }
+
+ String trinoInstance = getClusterNameFromQualifiedName(qualifiedName);
+ if (StringUtils.equals(resourceStr, qualifiedName)){
+ trinoInstance = resourceStr;
+ }
+
+ if (StringUtils.isEmpty(trinoInstance)) {
+ throwExceptionWithMessage("trino-instance not found in attribute '" + ENTITY_ATTRIBUTE_QUALIFIED_NAME + "': " + qualifiedName);
+ }
+ String serviceName = getRangerServiceName(trinoInstance);
+
+ String[] parts = resourceStr.split(QUALIFIED_NAME_DELIMITER);
+ if (parts.length < 1 || parts.length > 4) {
+ throwExceptionWithMessage("invalid resource format in attribute '" + ENTITY_ATTRIBUTE_QUALIFIED_NAME + "': " + qualifiedName);
+ }
+
+ Map<String, RangerPolicyResource> elements = new HashMap<>();
+ if (StringUtils.equals(entityType, ENTITY_TYPE_TRINO_INSTANCE) && StringUtils.equals(resourceStr, qualifiedName)) {
+ elements.put(ENTITY_TYPE_TRINO_INSTANCE, new RangerPolicyResource(resourceStr)); // there is no mapping for trino_instance in Ranger
Review Comment:
Entity type `ENTITY_TYPE_TRINO_INSTANCE` doesn't map to any resource type in Ranger Trino service-def. Notifications on this entity type should be ignored.
Consider excluding `ENTITY_TYPE_TRINO_INSTANCE` from `SUPPORTED_ENTITY_TYPES` so that notifications on this entity type will be ignored, even before `buildResource()` is called.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] RANGER-5522: update tagsync to process Atlas notifications for Trino entities [ranger]
Posted by "kumaab (via GitHub)" <gi...@apache.org>.
kumaab merged PR #895:
URL: https://github.com/apache/ranger/pull/895
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] RANGER-5522: update tagsync to process Atlas notifications for Trino entities [ranger]
Posted by "kumaab (via GitHub)" <gi...@apache.org>.
kumaab commented on code in PR #895:
URL: https://github.com/apache/ranger/pull/895#discussion_r3003627980
##########
tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasTrinoResourceMapper.java:
##########
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ranger.tagsync.source.atlas;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyResource;
+import org.apache.ranger.plugin.model.RangerServiceResource;
+import org.apache.ranger.tagsync.source.atlasrest.RangerAtlasEntity;
+
+import java.util.HashMap;
+import java.util.Map;
+
+public class AtlasTrinoResourceMapper extends AtlasResourceMapper {
+ public static final String ENTITY_TYPE_TRINO_INSTANCE = "trino_instance";
+ public static final String ENTITY_TYPE_TRINO_CATALOG = "trino_catalog";
+ public static final String ENTITY_TYPE_TRINO_SCHEMA = "trino_schema";
+ public static final String ENTITY_TYPE_TRINO_TABLE = "trino_table";
+ public static final String ENTITY_TYPE_TRINO_COLUMN = "trino_column";
+ public static final String RANGER_TYPE_TRINO_CATALOG = "catalog";
+ public static final String RANGER_TYPE_TRINO_SCHEMA = "schema";
+ public static final String RANGER_TYPE_TRINO_TABLE = "table";
+ public static final String RANGER_TYPE_TRINO_COLUMN = "column";
+
+ public static final String[] SUPPORTED_ENTITY_TYPES = {
+ ENTITY_TYPE_TRINO_INSTANCE,
+ ENTITY_TYPE_TRINO_CATALOG,
+ ENTITY_TYPE_TRINO_SCHEMA,
+ ENTITY_TYPE_TRINO_TABLE,
+ ENTITY_TYPE_TRINO_COLUMN
+ };
+
+ public AtlasTrinoResourceMapper() {
+ super("trino", SUPPORTED_ENTITY_TYPES);
+ }
+
+ /*
+ * qualifiedName can be of format, depending upon the entity-type:
+ * trino_instance: <instanceName>
+ * trino_catalog: <catalog>@<instanceName>
+ * trino_schema: <catalog>.<schema>@<instanceName>
+ * trino_table: <catalog>.<schema>.<table>@<instanceName>
+ * trino_column: <catalog>.<schema>.<table>.<column>@<instanceName>
+ */
+ @Override
+ public RangerServiceResource buildResource(RangerAtlasEntity entity) throws Exception {
+ String qualifiedName = (String) entity.getAttributes().get(AtlasResourceMapper.ENTITY_ATTRIBUTE_QUALIFIED_NAME);
+
+ if (StringUtils.isEmpty(qualifiedName)) {
+ throw new Exception("attribute '" + ENTITY_ATTRIBUTE_QUALIFIED_NAME + "' not found in entity");
+ }
+
+ String entityType = entity.getTypeName();
+ String entityGuid = entity.getGuid();
+ String resourceStr = getResourceNameFromQualifiedName(qualifiedName);
+
+ if (StringUtils.isEmpty(resourceStr)) {
+ throwExceptionWithMessage("resource not found in attribute '" + ENTITY_ATTRIBUTE_QUALIFIED_NAME + "': " + qualifiedName);
+ }
+
+ String trinoInstance = getClusterNameFromQualifiedName(qualifiedName);
+ if (StringUtils.equals(resourceStr, qualifiedName)){
+ trinoInstance = resourceStr;
+ }
+
+ if (StringUtils.isEmpty(trinoInstance)) {
+ throwExceptionWithMessage("trino-instance not found in attribute '" + ENTITY_ATTRIBUTE_QUALIFIED_NAME + "': " + qualifiedName);
+ }
+ String serviceName = getRangerServiceName(trinoInstance);
+
+ String[] parts = resourceStr.split(QUALIFIED_NAME_DELIMITER);
+ if (parts.length < 1 || parts.length > 4) {
+ throwExceptionWithMessage("invalid resource format in attribute '" + ENTITY_ATTRIBUTE_QUALIFIED_NAME + "': " + qualifiedName);
+ }
+
+ Map<String, RangerPolicyResource> elements = new HashMap<>();
+ if (StringUtils.equals(entityType, ENTITY_TYPE_TRINO_INSTANCE) && StringUtils.equals(resourceStr, qualifiedName)) {
+ elements.put(ENTITY_TYPE_TRINO_INSTANCE, new RangerPolicyResource(resourceStr)); // there is no mapping for trino_instance in Ranger
Review Comment:
Verified by sending a message for `trino_instance` with latest PR changes:
```
27 Mar 2026 23:12:11 DEBUG o.a.r.t.s.a.AtlasResourceMapperUtil [Thread-7] - 43 ==> isEntityTypeHandled(entityTypeName=trino_instance)
27 Mar 2026 23:12:11 DEBUG o.a.r.t.s.a.AtlasResourceMapperUtil [Thread-7] - 49 <== isEntityTypeHandled(entityTypeName=trino_instance) : false
27 Mar 2026 23:12:11 DEBUG o.a.r.t.s.a.AtlasTagSource [Thread-7] - 194 Message-offset=4, Notification={ Notification-Type: CLASSIFICATION_ADD, {entity={typeName=trino_instance, guid=11111111-1111-1111-1111-111111111111, attributes={ name=qualifiedName, value=dev}, , classifications={classificationName=SENSITIVE, attributes={{attributeName=kind,attributeValue=instance}}, validityPeriods={}}}}
27 Mar 2026 23:12:11 DEBUG o.a.r.t.s.a.AtlasNotificationMapper [Thread-7] - 128 Notification : [org.apache.ranger.tagsync.source.atlas.EntityNotificationWrapper@76f95b38] will NOT be processed.
27 Mar 2026 23:12:11 WARN o.a.r.t.s.a.AtlasNotificationMapper [Thread-7] - 72 Tag-sync is not enabled to handle notifications for Entity-type:[trino_instance]
27 Mar 2026 23:12:11 DEBUG o.a.r.t.s.a.AtlasNotificationMapper [Thread-7] - 75 Dropped process entity notification for Atlas-Entity [{typeName=trino_instance, guid=11111111-1111-1111-1111-111111111111, attributes={ name=qualifiedName, value=dev}]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] RANGER-5522: update tagsync to process Atlas notifications for Trino entities [ranger]
Posted by "mneethiraj (via GitHub)" <gi...@apache.org>.
mneethiraj commented on code in PR #895:
URL: https://github.com/apache/ranger/pull/895#discussion_r3003676463
##########
tagsync/src/test/java/org/apache/ranger/tagsync/process/TestTrinoResourceMapper.java:
##########
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ranger.tagsync.process;
+
+import org.apache.ranger.plugin.model.RangerServiceResource;
+import org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper;
+import org.apache.ranger.tagsync.source.atlasrest.RangerAtlasEntity;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.util.Collections;
+
+import static org.apache.ranger.tagsync.source.atlas.AtlasResourceMapper.ENTITY_ATTRIBUTE_QUALIFIED_NAME;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.ENTITY_TYPE_TRINO_CATALOG;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.ENTITY_TYPE_TRINO_COLUMN;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.ENTITY_TYPE_TRINO_INSTANCE;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.ENTITY_TYPE_TRINO_SCHEMA;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.ENTITY_TYPE_TRINO_TABLE;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.RANGER_TYPE_TRINO_CATALOG;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.RANGER_TYPE_TRINO_COLUMN;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.RANGER_TYPE_TRINO_SCHEMA;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.RANGER_TYPE_TRINO_TABLE;
+
+public class TestTrinoResourceMapper {
+ private static final String INSTANCE_QUALIFIED_NAME = "dev";
+ private static final String CATALOG_QUALIFIED_NAME = "sales@dev";
+ private static final String SCHEMA_QUALIFIED_NAME = "sales.reporting@dev";
+ private static final String TABLE_QUALIFIED_NAME = "sales.reporting.orders@dev";
+ private static final String COLUMN_QUALIFIED_NAME = "sales.reporting.orders.customer_id@dev";
+ private static final String INVALID_RESOURCE_QUALIFIED_NAME = "sales.reporting.orders.customer_id.extra@dev";
+
+ private static final String SERVICE_NAME = "dev_trino";
+ private static final String RANGER_CATALOG = "sales";
+ private static final String RANGER_SCHEMA = "reporting";
+ private static final String RANGER_TABLE = "orders";
+ private static final String RANGER_COLUMN = "customer_id";
+
+ AtlasTrinoResourceMapper resourceMapper = new AtlasTrinoResourceMapper();
+
+ @Test
+ public void testTrinoInstance() throws Exception {
Review Comment:
Tests for `ENTITY_TYPE_TRINO_INSTANCE` can either be removed or modified, as the mapper doesn't handle this entity type.
##########
tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasTrinoResourceMapper.java:
##########
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ranger.tagsync.source.atlas;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyResource;
+import org.apache.ranger.plugin.model.RangerServiceResource;
+import org.apache.ranger.tagsync.source.atlasrest.RangerAtlasEntity;
+
+import java.util.HashMap;
+import java.util.Map;
+
+public class AtlasTrinoResourceMapper extends AtlasResourceMapper {
+ public static final String ENTITY_TYPE_TRINO_INSTANCE = "trino_instance";
+ public static final String ENTITY_TYPE_TRINO_CATALOG = "trino_catalog";
+ public static final String ENTITY_TYPE_TRINO_SCHEMA = "trino_schema";
+ public static final String ENTITY_TYPE_TRINO_TABLE = "trino_table";
+ public static final String ENTITY_TYPE_TRINO_COLUMN = "trino_column";
+ public static final String RANGER_TYPE_TRINO_CATALOG = "catalog";
+ public static final String RANGER_TYPE_TRINO_SCHEMA = "schema";
+ public static final String RANGER_TYPE_TRINO_TABLE = "table";
+ public static final String RANGER_TYPE_TRINO_COLUMN = "column";
+
+ public static final String[] SUPPORTED_ENTITY_TYPES = {
+ ENTITY_TYPE_TRINO_CATALOG,
+ ENTITY_TYPE_TRINO_SCHEMA,
+ ENTITY_TYPE_TRINO_TABLE,
+ ENTITY_TYPE_TRINO_COLUMN
+ };
+
+ public AtlasTrinoResourceMapper() {
+ super("trino", SUPPORTED_ENTITY_TYPES);
+ }
+
+ /*
+ * qualifiedName can be of format, depending upon the entity-type:
+ * trino_instance: <instanceName>
+ * trino_catalog: <catalog>@<instanceName>
+ * trino_schema: <catalog>.<schema>@<instanceName>
+ * trino_table: <catalog>.<schema>.<table>@<instanceName>
+ * trino_column: <catalog>.<schema>.<table>.<column>@<instanceName>
+ */
+ @Override
+ public RangerServiceResource buildResource(RangerAtlasEntity entity) throws Exception {
+ String qualifiedName = (String) entity.getAttributes().get(AtlasResourceMapper.ENTITY_ATTRIBUTE_QUALIFIED_NAME);
+
+ if (StringUtils.isEmpty(qualifiedName)) {
+ throw new Exception("attribute '" + ENTITY_ATTRIBUTE_QUALIFIED_NAME + "' not found in entity");
+ }
+
+ String entityType = entity.getTypeName();
+ String entityGuid = entity.getGuid();
+ String resourceStr = getResourceNameFromQualifiedName(qualifiedName);
+
+ if (StringUtils.isEmpty(resourceStr)) {
+ throwExceptionWithMessage("resource not found in attribute '" + ENTITY_ATTRIBUTE_QUALIFIED_NAME + "': " + qualifiedName);
+ }
+
+ String trinoInstance = getClusterNameFromQualifiedName(qualifiedName);
+ if (StringUtils.equals(resourceStr, qualifiedName)){
+ trinoInstance = resourceStr;
+ }
+
+ if (StringUtils.isEmpty(trinoInstance)) {
+ throwExceptionWithMessage("trino-instance not found in attribute '" + ENTITY_ATTRIBUTE_QUALIFIED_NAME + "': " + qualifiedName);
+ }
+ String serviceName = getRangerServiceName(trinoInstance);
+
+ String[] parts = resourceStr.split(QUALIFIED_NAME_DELIMITER);
+ if (parts.length < 1 || parts.length > 4) {
+ throwExceptionWithMessage("invalid resource format in attribute '" + ENTITY_ATTRIBUTE_QUALIFIED_NAME + "': " + qualifiedName);
+ }
+
+ Map<String, RangerPolicyResource> elements = new HashMap<>();
+ if (StringUtils.equals(entityType, ENTITY_TYPE_TRINO_INSTANCE) && StringUtils.equals(resourceStr, qualifiedName)) {
Review Comment:
Lines 92 & 93, and 31 can be removed, since entity type `ENTITY_TYPE_TRINO_INSTANCE` is not handled anymore.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] RANGER-5522: update tagsync to process Atlas notifications for Trino entities [ranger]
Posted by "mneethiraj (via GitHub)" <gi...@apache.org>.
mneethiraj commented on code in PR #895:
URL: https://github.com/apache/ranger/pull/895#discussion_r3001749267
##########
tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasTrinoResourceMapper.java:
##########
@@ -0,0 +1,123 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ranger.tagsync.source.atlas;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyResource;
+import org.apache.ranger.plugin.model.RangerServiceResource;
+import org.apache.ranger.tagsync.source.atlasrest.RangerAtlasEntity;
+
+import java.util.HashMap;
+import java.util.Map;
+
+public class AtlasTrinoResourceMapper extends AtlasResourceMapper {
+ public static final String ENTITY_TYPE_TRINO_INSTANCE = "trino_instance";
+ public static final String ENTITY_TYPE_TRINO_CATALOG = "trino_catalog";
+ public static final String ENTITY_TYPE_TRINO_SCHEMA = "trino_schema";
+ public static final String ENTITY_TYPE_TRINO_TABLE = "trino_table";
+ public static final String ENTITY_TYPE_TRINO_COLUMN = "trino_column";
+ public static final String RANGER_TYPE_TRINO_CATALOG = "catalog";
+ public static final String RANGER_TYPE_TRINO_SCHEMA = "schema";
+ public static final String RANGER_TYPE_TRINO_TABLE = "table";
+ public static final String RANGER_TYPE_TRINO_COLUMN = "column";
+
+ public static final String[] SUPPORTED_ENTITY_TYPES = {
+ ENTITY_TYPE_TRINO_INSTANCE,
+ ENTITY_TYPE_TRINO_CATALOG,
+ ENTITY_TYPE_TRINO_SCHEMA,
+ ENTITY_TYPE_TRINO_TABLE,
+ ENTITY_TYPE_TRINO_COLUMN
+ };
+
+ public AtlasTrinoResourceMapper() {
+ super("trino", SUPPORTED_ENTITY_TYPES);
+ }
+
+ /*
+ * qualifiedName can be of format, depending upon the entity-type:
+ * trino_instance: <instanceName>
+ * trino_catalog: <catalog>@<instanceName>
+ * trino_schema: <catalog>.<schema>@<instanceName>
+ * trino_table: <catalog>.<schema>.<table>@<instanceName>
+ * trino_column: <catalog>.<schema>.<table>.<column>@<instanceName>
+ */
+ @Override
+ public RangerServiceResource buildResource(RangerAtlasEntity entity) throws Exception {
+ String qualifiedName = (String) entity.getAttributes().get(AtlasResourceMapper.ENTITY_ATTRIBUTE_QUALIFIED_NAME);
+
+ if (StringUtils.isEmpty(qualifiedName)) {
+ throw new Exception("attribute '" + ENTITY_ATTRIBUTE_QUALIFIED_NAME + "' not found in entity");
+ }
+
+ String entityType = entity.getTypeName();
+ String entityGuid = entity.getGuid();
+ String resourceStr = getResourceNameFromQualifiedName(qualifiedName);
+
+ if (StringUtils.isEmpty(resourceStr)) {
+ throwExceptionWithMessage("resource not found in attribute '" + ENTITY_ATTRIBUTE_QUALIFIED_NAME + "': " + qualifiedName);
+ }
+
+ String trinoInstance = getClusterNameFromQualifiedName(qualifiedName);
+ if (StringUtils.isEmpty(trinoInstance)) {
Review Comment:
Looking at qualified name for `trino_instance` at line 55 above (and line 90 below), `trinoInstance` here will be empty for entity type `trino_instance`, right?
##########
tagsync/src/test/java/org/apache/ranger/tagsync/process/TestTrinoResourceMapper.java:
##########
@@ -0,0 +1,185 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ranger.tagsync.process;
+
+import org.apache.ranger.plugin.model.RangerServiceResource;
+import org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper;
+import org.apache.ranger.tagsync.source.atlasrest.RangerAtlasEntity;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.util.Collections;
+
+import static org.apache.ranger.tagsync.source.atlas.AtlasResourceMapper.ENTITY_ATTRIBUTE_QUALIFIED_NAME;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.ENTITY_TYPE_TRINO_CATALOG;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.ENTITY_TYPE_TRINO_COLUMN;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.ENTITY_TYPE_TRINO_INSTANCE;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.ENTITY_TYPE_TRINO_SCHEMA;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.ENTITY_TYPE_TRINO_TABLE;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.RANGER_TYPE_TRINO_CATALOG;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.RANGER_TYPE_TRINO_COLUMN;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.RANGER_TYPE_TRINO_SCHEMA;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.RANGER_TYPE_TRINO_TABLE;
+
+public class TestTrinoResourceMapper {
+ private static final String INSTANCE_QUALIFIED_NAME = "dev";
+ private static final String CATALOG_QUALIFIED_NAME = "sales@dev";
+ private static final String SCHEMA_QUALIFIED_NAME = "sales.reporting@dev";
+ private static final String TABLE_QUALIFIED_NAME = "sales.reporting.orders@dev";
+ private static final String COLUMN_QUALIFIED_NAME = "sales.reporting.orders.customer_id@dev";
+ private static final String INVALID_RESOURCE_QUALIFIED_NAME = "sales.reporting.orders.customer_id.extra@dev";
+
+ private static final String SERVICE_NAME = "dev_trino";
+ private static final String RANGER_CATALOG = "sales";
+ private static final String RANGER_SCHEMA = "reporting";
+ private static final String RANGER_TABLE = "orders";
+ private static final String RANGER_COLUMN = "customer_id";
+
+ AtlasTrinoResourceMapper resourceMapper = new AtlasTrinoResourceMapper();
+
+ @Test
Review Comment:
Add tests for `trino_instance` type as well - `testTrinoInstance()`.
##########
tagsync/src/test/java/org/apache/ranger/tagsync/process/TestTrinoResourceMapper.java:
##########
@@ -0,0 +1,185 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ranger.tagsync.process;
+
+import org.apache.ranger.plugin.model.RangerServiceResource;
+import org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper;
+import org.apache.ranger.tagsync.source.atlasrest.RangerAtlasEntity;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.util.Collections;
+
+import static org.apache.ranger.tagsync.source.atlas.AtlasResourceMapper.ENTITY_ATTRIBUTE_QUALIFIED_NAME;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.ENTITY_TYPE_TRINO_CATALOG;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.ENTITY_TYPE_TRINO_COLUMN;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.ENTITY_TYPE_TRINO_INSTANCE;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.ENTITY_TYPE_TRINO_SCHEMA;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.ENTITY_TYPE_TRINO_TABLE;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.RANGER_TYPE_TRINO_CATALOG;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.RANGER_TYPE_TRINO_COLUMN;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.RANGER_TYPE_TRINO_SCHEMA;
+import static org.apache.ranger.tagsync.source.atlas.AtlasTrinoResourceMapper.RANGER_TYPE_TRINO_TABLE;
+
+public class TestTrinoResourceMapper {
+ private static final String INSTANCE_QUALIFIED_NAME = "dev";
+ private static final String CATALOG_QUALIFIED_NAME = "sales@dev";
+ private static final String SCHEMA_QUALIFIED_NAME = "sales.reporting@dev";
+ private static final String TABLE_QUALIFIED_NAME = "sales.reporting.orders@dev";
+ private static final String COLUMN_QUALIFIED_NAME = "sales.reporting.orders.customer_id@dev";
+ private static final String INVALID_RESOURCE_QUALIFIED_NAME = "sales.reporting.orders.customer_id.extra@dev";
+
+ private static final String SERVICE_NAME = "dev_trino";
+ private static final String RANGER_CATALOG = "sales";
+ private static final String RANGER_SCHEMA = "reporting";
+ private static final String RANGER_TABLE = "orders";
+ private static final String RANGER_COLUMN = "customer_id";
+
+ AtlasTrinoResourceMapper resourceMapper = new AtlasTrinoResourceMapper();
+
+ @Test
+ public void testTrinoCatalog() throws Exception {
+ RangerAtlasEntity entity = getEntity(ENTITY_TYPE_TRINO_CATALOG, CATALOG_QUALIFIED_NAME);
+ RangerServiceResource resource = resourceMapper.buildResource(entity);
+
+ Assertions.assertEquals(SERVICE_NAME, resource.getServiceName());
+ assertCatalogResource(resource);
+ }
+
+ @Test
+ public void testTrinoSchema() throws Exception {
+ RangerAtlasEntity entity = getEntity(ENTITY_TYPE_TRINO_SCHEMA, SCHEMA_QUALIFIED_NAME);
+ RangerServiceResource resource = resourceMapper.buildResource(entity);
+
+ Assertions.assertEquals(SERVICE_NAME, resource.getServiceName());
+ assertSchemaResource(resource);
+ }
+
+ @Test
+ public void testTrinoTable() throws Exception {
+ RangerAtlasEntity entity = getEntity(ENTITY_TYPE_TRINO_TABLE, TABLE_QUALIFIED_NAME);
+ RangerServiceResource resource = resourceMapper.buildResource(entity);
+
+ Assertions.assertEquals(SERVICE_NAME, resource.getServiceName());
+ assertTableResource(resource);
+ }
+
+ @Test
+ public void testTrinoColumn() throws Exception {
+ RangerAtlasEntity entity = getEntity(ENTITY_TYPE_TRINO_COLUMN, COLUMN_QUALIFIED_NAME);
+ RangerServiceResource resource = resourceMapper.buildResource(entity);
+
+ Assertions.assertEquals(SERVICE_NAME, resource.getServiceName());
+ assertColumnResource(resource);
+ }
+
+ @Test
+ public void testInvalidCatalogEntity() {
+ assertException(getEntity(ENTITY_TYPE_TRINO_CATALOG, null), "attribute 'qualifiedName' not found");
+ assertException(getEntity(ENTITY_TYPE_TRINO_CATALOG, ""), "attribute 'qualifiedName' not found");
+ assertException(getEntity(ENTITY_TYPE_TRINO_CATALOG, "sales"), "trino-instance not found");
+ }
+
+ @Test
+ public void testInvalidSchemaEntity() {
+ assertException(getEntity(ENTITY_TYPE_TRINO_SCHEMA, null), "attribute 'qualifiedName' not found");
+ assertException(getEntity(ENTITY_TYPE_TRINO_SCHEMA, ""), "attribute 'qualifiedName' not found");
+ assertException(getEntity(ENTITY_TYPE_TRINO_SCHEMA, "sales.reporting"), "trino-instance not found");
+ assertException(getEntity(ENTITY_TYPE_TRINO_SCHEMA, CATALOG_QUALIFIED_NAME), "invalid qualifiedName");
+ }
+
+ @Test
+ public void testInvalidTableEntity() {
+ assertException(getEntity(ENTITY_TYPE_TRINO_TABLE, null), "attribute 'qualifiedName' not found");
+ assertException(getEntity(ENTITY_TYPE_TRINO_TABLE, ""), "attribute 'qualifiedName' not found");
+ assertException(getEntity(ENTITY_TYPE_TRINO_TABLE, "sales.reporting.orders"), "trino-instance not found");
+ assertException(getEntity(ENTITY_TYPE_TRINO_TABLE, SCHEMA_QUALIFIED_NAME), "invalid qualifiedName");
+ }
+
+ @Test
+ public void testInvalidColumnEntity() {
+ assertException(getEntity(ENTITY_TYPE_TRINO_COLUMN, null), "attribute 'qualifiedName' not found");
+ assertException(getEntity(ENTITY_TYPE_TRINO_COLUMN, ""), "attribute 'qualifiedName' not found");
+ assertException(getEntity(ENTITY_TYPE_TRINO_COLUMN, "sales.reporting.orders.customer_id"), "trino-instance not found");
+ assertException(getEntity(ENTITY_TYPE_TRINO_COLUMN, TABLE_QUALIFIED_NAME), "invalid qualifiedName");
+ assertException(getEntity(ENTITY_TYPE_TRINO_COLUMN, INVALID_RESOURCE_QUALIFIED_NAME), "invalid resource format");
+ }
+
+ @Test
+ public void testInvalidInstanceEntity() {
+ assertException(getEntity(ENTITY_TYPE_TRINO_INSTANCE, null), "attribute 'qualifiedName' not found");
+ assertException(getEntity(ENTITY_TYPE_TRINO_INSTANCE, ""), "attribute 'qualifiedName' not found");
+ assertException(getEntity(ENTITY_TYPE_TRINO_INSTANCE, INSTANCE_QUALIFIED_NAME), "trino-instance not found");
+ }
+
+ private RangerAtlasEntity getEntity(String entityType, String qualifiedName) {
+ return new RangerAtlasEntity(entityType, "guid-" + entityType, Collections.singletonMap(ENTITY_ATTRIBUTE_QUALIFIED_NAME, qualifiedName));
+ }
+
+ private void assertResourceElementCount(RangerServiceResource resource, int count) {
+ Assertions.assertNotNull(resource);
+ Assertions.assertEquals(SERVICE_NAME, resource.getServiceName());
+ Assertions.assertNotNull(resource.getResourceElements());
+ Assertions.assertEquals(count, resource.getResourceElements().size());
+ }
+
+ private void assertCatalogResource(RangerServiceResource resource) {
+ assertResourceElementCount(resource, 1);
+ assertResourceElementValue(resource, RANGER_TYPE_TRINO_CATALOG, RANGER_CATALOG);
+ }
+
+ private void assertSchemaResource(RangerServiceResource resource) {
+ assertResourceElementCount(resource, 2);
+ assertResourceElementValue(resource, RANGER_TYPE_TRINO_CATALOG, RANGER_CATALOG);
+ assertResourceElementValue(resource, RANGER_TYPE_TRINO_SCHEMA, RANGER_SCHEMA);
+ }
+
+ private void assertTableResource(RangerServiceResource resource) {
+ assertResourceElementCount(resource, 3);
+ assertResourceElementValue(resource, RANGER_TYPE_TRINO_CATALOG, RANGER_CATALOG);
+ assertResourceElementValue(resource, RANGER_TYPE_TRINO_SCHEMA, RANGER_SCHEMA);
+ assertResourceElementValue(resource, RANGER_TYPE_TRINO_TABLE, RANGER_TABLE);
+ }
+
+ private void assertColumnResource(RangerServiceResource resource) {
+ assertResourceElementCount(resource, 4);
+ assertResourceElementValue(resource, RANGER_TYPE_TRINO_CATALOG, RANGER_CATALOG);
+ assertResourceElementValue(resource, RANGER_TYPE_TRINO_SCHEMA, RANGER_SCHEMA);
+ assertResourceElementValue(resource, RANGER_TYPE_TRINO_TABLE, RANGER_TABLE);
+ assertResourceElementValue(resource, RANGER_TYPE_TRINO_COLUMN, RANGER_COLUMN);
+ }
+
+ private void assertResourceElementValue(RangerServiceResource resource, String resourceName, String value) {
+ Assertions.assertTrue(resource.getResourceElements().containsKey(resourceName));
+ Assertions.assertNotNull(resource.getResourceElements().get(resourceName).getValues());
+ Assertions.assertEquals(1, resource.getResourceElements().get(resourceName).getValues().size());
+ Assertions.assertEquals(value, resource.getResourceElements().get(resourceName).getValues().get(0));
+ }
+
+ private void assertException(RangerAtlasEntity entity, String exceptionMessage) {
+ try {
+ RangerServiceResource resource = resourceMapper.buildResource(entity);
+
+ Assertions.assertFalse(true, "Expected buildResource() to fail. But it returned " + resource);
Review Comment:
Replace `Assertions.assertFalse(true, ` with `Assertions.fail(`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] RANGER-5522: update tagsync to process Atlas notifications for Trino entities [ranger]
Posted by "kumaab (via GitHub)" <gi...@apache.org>.
kumaab commented on code in PR #895:
URL: https://github.com/apache/ranger/pull/895#discussion_r3002564799
##########
tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasTrinoResourceMapper.java:
##########
@@ -0,0 +1,123 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ranger.tagsync.source.atlas;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyResource;
+import org.apache.ranger.plugin.model.RangerServiceResource;
+import org.apache.ranger.tagsync.source.atlasrest.RangerAtlasEntity;
+
+import java.util.HashMap;
+import java.util.Map;
+
+public class AtlasTrinoResourceMapper extends AtlasResourceMapper {
+ public static final String ENTITY_TYPE_TRINO_INSTANCE = "trino_instance";
+ public static final String ENTITY_TYPE_TRINO_CATALOG = "trino_catalog";
+ public static final String ENTITY_TYPE_TRINO_SCHEMA = "trino_schema";
+ public static final String ENTITY_TYPE_TRINO_TABLE = "trino_table";
+ public static final String ENTITY_TYPE_TRINO_COLUMN = "trino_column";
+ public static final String RANGER_TYPE_TRINO_CATALOG = "catalog";
+ public static final String RANGER_TYPE_TRINO_SCHEMA = "schema";
+ public static final String RANGER_TYPE_TRINO_TABLE = "table";
+ public static final String RANGER_TYPE_TRINO_COLUMN = "column";
+
+ public static final String[] SUPPORTED_ENTITY_TYPES = {
+ ENTITY_TYPE_TRINO_INSTANCE,
+ ENTITY_TYPE_TRINO_CATALOG,
+ ENTITY_TYPE_TRINO_SCHEMA,
+ ENTITY_TYPE_TRINO_TABLE,
+ ENTITY_TYPE_TRINO_COLUMN
+ };
+
+ public AtlasTrinoResourceMapper() {
+ super("trino", SUPPORTED_ENTITY_TYPES);
+ }
+
+ /*
+ * qualifiedName can be of format, depending upon the entity-type:
+ * trino_instance: <instanceName>
+ * trino_catalog: <catalog>@<instanceName>
+ * trino_schema: <catalog>.<schema>@<instanceName>
+ * trino_table: <catalog>.<schema>.<table>@<instanceName>
+ * trino_column: <catalog>.<schema>.<table>.<column>@<instanceName>
+ */
+ @Override
+ public RangerServiceResource buildResource(RangerAtlasEntity entity) throws Exception {
+ String qualifiedName = (String) entity.getAttributes().get(AtlasResourceMapper.ENTITY_ATTRIBUTE_QUALIFIED_NAME);
+
+ if (StringUtils.isEmpty(qualifiedName)) {
+ throw new Exception("attribute '" + ENTITY_ATTRIBUTE_QUALIFIED_NAME + "' not found in entity");
+ }
+
+ String entityType = entity.getTypeName();
+ String entityGuid = entity.getGuid();
+ String resourceStr = getResourceNameFromQualifiedName(qualifiedName);
+
+ if (StringUtils.isEmpty(resourceStr)) {
+ throwExceptionWithMessage("resource not found in attribute '" + ENTITY_ATTRIBUTE_QUALIFIED_NAME + "': " + qualifiedName);
+ }
+
+ String trinoInstance = getClusterNameFromQualifiedName(qualifiedName);
+ if (StringUtils.isEmpty(trinoInstance)) {
Review Comment:
Yeah, fixed it, thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] RANGER-5522: update tagsync to process Atlas notifications for Trino entities [ranger]
Posted by "kumaab (via GitHub)" <gi...@apache.org>.
kumaab commented on code in PR #895:
URL: https://github.com/apache/ranger/pull/895#discussion_r3003814033
##########
tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasTrinoResourceMapper.java:
##########
@@ -0,0 +1,123 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ranger.tagsync.source.atlas;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.ranger.plugin.model.RangerPolicy.RangerPolicyResource;
+import org.apache.ranger.plugin.model.RangerServiceResource;
+import org.apache.ranger.tagsync.source.atlasrest.RangerAtlasEntity;
+
+import java.util.HashMap;
+import java.util.Map;
+
+public class AtlasTrinoResourceMapper extends AtlasResourceMapper {
+ public static final String ENTITY_TYPE_TRINO_CATALOG = "trino_catalog";
+ public static final String ENTITY_TYPE_TRINO_SCHEMA = "trino_schema";
+ public static final String ENTITY_TYPE_TRINO_TABLE = "trino_table";
+ public static final String ENTITY_TYPE_TRINO_COLUMN = "trino_column";
+ public static final String RANGER_TYPE_TRINO_CATALOG = "catalog";
+ public static final String RANGER_TYPE_TRINO_SCHEMA = "schema";
+ public static final String RANGER_TYPE_TRINO_TABLE = "table";
+ public static final String RANGER_TYPE_TRINO_COLUMN = "column";
+
+ public static final String[] SUPPORTED_ENTITY_TYPES = {
+ ENTITY_TYPE_TRINO_CATALOG,
+ ENTITY_TYPE_TRINO_SCHEMA,
+ ENTITY_TYPE_TRINO_TABLE,
+ ENTITY_TYPE_TRINO_COLUMN
+ };
+
+ public AtlasTrinoResourceMapper() {
+ super("trino", SUPPORTED_ENTITY_TYPES);
+ }
+
+ /*
+ * qualifiedName can be of format, depending upon the entity-type:
+ * trino_instance: <instanceName>
Review Comment:
done
##########
tagsync/conf.dist/logback.xml:
##########
@@ -31,6 +31,9 @@
<cleanHistoryOnStart>true</cleanHistoryOnStart>
</rollingPolicy>
</appender>
+ <logger name="org.apache.ranger" additivity="false" level="info"> <!-- toggle to debug for testing -->
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ranger.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org