You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nifi.apache.org by jo...@apache.org on 2023/10/12 00:45:46 UTC

[nifi] branch main updated (9627c4c0e8 -> 1a0f509ffc)

This is an automated email from the ASF dual-hosted git repository.

joewitt pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/nifi.git


    from 9627c4c0e8 NIFI-12174 Upgraded AWS PutLambda Processor to SDK 2
     new f2927525f3 NIFI-12213 This closes #7871. add nifi-utils dependency to NARs that do not use nifi-standard-services-api-nar as a parent
     new 584b3fc165 NIFI-12214 This closes #7869. ConsumeElasticsearch Query Builder properties do not dependOn the Query Definition Style
     new 1a0f509ffc NIFI-12211: This closes #7868. add Validator.VALID to username/password on TinkerpopClientService

The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../apache/nifi/components/PropertyDescriptor.java | 30 ++++++++++++++++++++
 .../nifi/components/TestPropertyDescriptor.java    | 33 ++++++++++++++++++++++
 .../elasticsearch/ConsumeElasticsearch.java        | 32 +++++++++++++++++++++
 .../elasticsearch/ConsumeElasticsearchTest.java    |  8 +++---
 .../integration/AbstractElasticsearchITBase.java   |  2 +-
 nifi-nar-bundles/nifi-elasticsearch-bundle/pom.xml |  4 +--
 .../apache/nifi/graph/TinkerpopClientService.java  |  2 ++
 .../nifi-kerberos-iaa-providers-nar/pom.xml        |  5 ++++
 .../nifi-ldap-iaa-providers-nar/pom.xml            |  5 ++++
 9 files changed, 114 insertions(+), 7 deletions(-)


[nifi] 01/03: NIFI-12213 This closes #7871. add nifi-utils dependency to NARs that do not use nifi-standard-services-api-nar as a parent

Posted by jo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

joewitt pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi.git

commit f2927525f30934c9566b0d0b400b268cac0e9297
Author: Mike Moser <mo...@apache.org>
AuthorDate: Wed Oct 11 20:38:46 2023 +0000

    NIFI-12213 This closes #7871. add nifi-utils dependency to NARs that do not use nifi-standard-services-api-nar as a parent
    
    Signed-off-by: Joseph Witt <jo...@apache.org>
---
 .../nifi-kerberos-iaa-providers-nar/pom.xml                          | 5 +++++
 .../nifi-ldap-iaa-providers-nar/pom.xml                              | 5 +++++
 2 files changed, 10 insertions(+)

diff --git a/nifi-nar-bundles/nifi-kerberos-iaa-providers-bundle/nifi-kerberos-iaa-providers-nar/pom.xml b/nifi-nar-bundles/nifi-kerberos-iaa-providers-bundle/nifi-kerberos-iaa-providers-nar/pom.xml
index 1faa60f814..4e3519dadc 100644
--- a/nifi-nar-bundles/nifi-kerberos-iaa-providers-bundle/nifi-kerberos-iaa-providers-nar/pom.xml
+++ b/nifi-nar-bundles/nifi-kerberos-iaa-providers-bundle/nifi-kerberos-iaa-providers-nar/pom.xml
@@ -37,6 +37,11 @@
             <artifactId>nifi-security-utils-api</artifactId>
             <scope>compile</scope>
         </dependency>
+        <dependency>
+            <groupId>org.apache.nifi</groupId>
+            <artifactId>nifi-utils</artifactId>
+            <scope>compile</scope>
+        </dependency>
     </dependencies>
     <name>nifi-kerberos-iaa-providers-nar</name>
 </project>
\ No newline at end of file
diff --git a/nifi-nar-bundles/nifi-ldap-iaa-providers-bundle/nifi-ldap-iaa-providers-nar/pom.xml b/nifi-nar-bundles/nifi-ldap-iaa-providers-bundle/nifi-ldap-iaa-providers-nar/pom.xml
index 0180c97403..44c47bfdb9 100644
--- a/nifi-nar-bundles/nifi-ldap-iaa-providers-bundle/nifi-ldap-iaa-providers-nar/pom.xml
+++ b/nifi-nar-bundles/nifi-ldap-iaa-providers-bundle/nifi-ldap-iaa-providers-nar/pom.xml
@@ -37,6 +37,11 @@
             <artifactId>nifi-security-utils-api</artifactId>
             <scope>compile</scope>
         </dependency>
+        <dependency>
+            <groupId>org.apache.nifi</groupId>
+            <artifactId>nifi-utils</artifactId>
+            <scope>compile</scope>
+        </dependency>
     </dependencies>
     <name>nifi-ldap-iaa-providers-nar</name>
 </project>
\ No newline at end of file


[nifi] 02/03: NIFI-12214 This closes #7869. ConsumeElasticsearch Query Builder properties do not dependOn the Query Definition Style

Posted by jo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

joewitt pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi.git

commit 584b3fc1652809e49f7d8f34602208ad20e6e1d0
Author: Chris Sampson <ch...@gmail.com>
AuthorDate: Wed Oct 11 21:00:09 2023 +0100

    NIFI-12214 This closes #7869. ConsumeElasticsearch Query Builder properties do not dependOn the Query Definition Style
    
    Signed-off-by: Joseph Witt <jo...@apache.org>
---
 .../apache/nifi/components/PropertyDescriptor.java | 30 ++++++++++++++++++++
 .../nifi/components/TestPropertyDescriptor.java    | 33 ++++++++++++++++++++++
 .../elasticsearch/ConsumeElasticsearch.java        | 32 +++++++++++++++++++++
 .../elasticsearch/ConsumeElasticsearchTest.java    |  8 +++---
 .../integration/AbstractElasticsearchITBase.java   |  2 +-
 nifi-nar-bundles/nifi-elasticsearch-bundle/pom.xml |  4 +--
 6 files changed, 102 insertions(+), 7 deletions(-)

diff --git a/nifi-api/src/main/java/org/apache/nifi/components/PropertyDescriptor.java b/nifi-api/src/main/java/org/apache/nifi/components/PropertyDescriptor.java
index be359544e4..48bc181f85 100644
--- a/nifi-api/src/main/java/org/apache/nifi/components/PropertyDescriptor.java
+++ b/nifi-api/src/main/java/org/apache/nifi/components/PropertyDescriptor.java
@@ -413,6 +413,16 @@ public final class PropertyDescriptor implements Comparable<PropertyDescriptor>
             return this;
         }
 
+        /**
+         * Clears all Allowable Values from this Property
+         *
+         * @return the builder
+         */
+        public Builder clearAllowableValues() {
+            this.allowableValues = null;
+            return this;
+        }
+
         /**
          * Sets the Allowable Values for this Property
          *
@@ -455,6 +465,16 @@ public final class PropertyDescriptor implements Comparable<PropertyDescriptor>
             return this;
         }
 
+        /**
+         * Clear all Validators from this Property
+         *
+         * @return the builder
+         */
+        public Builder clearValidators() {
+            validators.clear();
+            return this;
+        }
+
         /**
          * Specifies that this property provides the identifier of a Controller
          * Service that implements the given interface
@@ -607,6 +627,16 @@ public final class PropertyDescriptor implements Comparable<PropertyDescriptor>
             return dependsOn(property, dependentValues);
         }
 
+        /**
+         * Clear all Dependencies from this Property
+         *
+         * @return the builder
+         */
+        public Builder clearDependsOn() {
+            this.dependencies = new HashSet<>();
+            return this;
+        }
+
         private AllowableValue toAllowableValue(DescribedValue describedValue) {
             return new AllowableValue(describedValue.getValue(), describedValue.getDisplayName(), describedValue.getDescription());
         }
diff --git a/nifi-api/src/test/java/org/apache/nifi/components/TestPropertyDescriptor.java b/nifi-api/src/test/java/org/apache/nifi/components/TestPropertyDescriptor.java
index 7eeac9a61a..8e20a82e52 100644
--- a/nifi-api/src/test/java/org/apache/nifi/components/TestPropertyDescriptor.java
+++ b/nifi-api/src/test/java/org/apache/nifi/components/TestPropertyDescriptor.java
@@ -35,6 +35,7 @@ import java.util.stream.Stream;
 import static org.junit.jupiter.api.Assertions.assertEquals;
 import static org.junit.jupiter.api.Assertions.assertFalse;
 import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertNull;
 import static org.junit.jupiter.api.Assertions.assertThrows;
 import static org.junit.jupiter.api.Assertions.assertTrue;
 import static org.mockito.ArgumentMatchers.anyString;
@@ -188,4 +189,36 @@ public class TestPropertyDescriptor {
         // Test the literal value 'target'
         assertTrue(withElNotAllowed.validate("target", validationContext).isValid());
     }
+
+    @Test
+    void testClearingValues() {
+        final PropertyDescriptor dep1 = new PropertyDescriptor.Builder()
+                .name("dep1")
+                .allowableValues("delVal1")
+                .build();
+
+        final PropertyDescriptor pd1 = new PropertyDescriptor.Builder()
+                .name("test")
+                .addValidator(Validator.VALID)
+                .allowableValues("val1")
+                .dependsOn(dep1, "depVal1")
+                .build();
+
+        final PropertyDescriptor pd2 = new PropertyDescriptor.Builder()
+                .fromPropertyDescriptor(pd1)
+                .clearValidators()
+                .clearAllowableValues()
+                .clearDependsOn()
+                .build();
+
+        assertEquals("test", pd1.getName());
+        assertFalse(pd1.getValidators().isEmpty());
+        assertFalse(pd1.getDependencies().isEmpty());
+        assertNotNull(pd1.getAllowableValues());
+
+        assertEquals("test", pd2.getName());
+        assertTrue(pd2.getValidators().isEmpty());
+        assertTrue(pd2.getDependencies().isEmpty());
+        assertNull(pd2.getAllowableValues());
+    }
 }
diff --git a/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/main/java/org/apache/nifi/processors/elasticsearch/ConsumeElasticsearch.java b/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/main/java/org/apache/nifi/processors/elasticsearch/ConsumeElasticsearch.java
index c120471788..5f409784eb 100644
--- a/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/main/java/org/apache/nifi/processors/elasticsearch/ConsumeElasticsearch.java
+++ b/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/main/java/org/apache/nifi/processors/elasticsearch/ConsumeElasticsearch.java
@@ -85,6 +85,31 @@ import java.util.stream.Collectors;
 public class ConsumeElasticsearch extends SearchElasticsearch {
     static final String STATE_RANGE_VALUE = "trackingRangeValue";
 
+    public static final PropertyDescriptor SIZE = new PropertyDescriptor.Builder()
+            .fromPropertyDescriptor(ElasticsearchRestProcessor.SIZE)
+            .clearDependsOn() // always show the Query Builder properties for ConsumeElasticsearch
+            .build();
+
+    public static final PropertyDescriptor AGGREGATIONS = new PropertyDescriptor.Builder()
+            .fromPropertyDescriptor(ElasticsearchRestProcessor.AGGREGATIONS)
+            .clearDependsOn() // always show the Query Builder properties for ConsumeElasticsearch
+            .build();
+
+    public static final PropertyDescriptor SORT = new PropertyDescriptor.Builder()
+            .fromPropertyDescriptor(ElasticsearchRestProcessor.SORT)
+            .clearDependsOn() // always show the Query Builder properties for ConsumeElasticsearch
+            .build();
+
+    public static final PropertyDescriptor FIELDS = new PropertyDescriptor.Builder()
+            .fromPropertyDescriptor(ElasticsearchRestProcessor.FIELDS)
+            .clearDependsOn() // always show the Query Builder properties for ConsumeElasticsearch
+            .build();
+
+    public static final PropertyDescriptor SCRIPT_FIELDS = new PropertyDescriptor.Builder()
+            .fromPropertyDescriptor(ElasticsearchRestProcessor.SCRIPT_FIELDS)
+            .clearDependsOn() // always show the Query Builder properties for ConsumeElasticsearch
+            .build();
+
     public static final PropertyDescriptor RANGE_FIELD = new PropertyDescriptor.Builder()
             .name("es-rest-range-field")
             .displayName("Range Query Field")
@@ -163,6 +188,13 @@ public class ConsumeElasticsearch extends SearchElasticsearch {
                 .filter(pd -> !QUERY.equals(pd) && !QUERY_CLAUSE.equals(pd) && !QUERY_DEFINITION_STYLE.equals(pd))
                 .collect(Collectors.toList()));
 
+        // replace Query Builder properties with updated version without the property dependencies that are invalid for ConsumeElasticsearch
+        descriptors.set(descriptors.indexOf(ElasticsearchRestProcessor.SIZE), ConsumeElasticsearch.SIZE);
+        descriptors.set(descriptors.indexOf(ElasticsearchRestProcessor.AGGREGATIONS), ConsumeElasticsearch.AGGREGATIONS);
+        descriptors.set(descriptors.indexOf(ElasticsearchRestProcessor.SORT), ConsumeElasticsearch.SORT);
+        descriptors.set(descriptors.indexOf(ElasticsearchRestProcessor.FIELDS), ConsumeElasticsearch.FIELDS);
+        descriptors.set(descriptors.indexOf(ElasticsearchRestProcessor.SCRIPT_FIELDS), ConsumeElasticsearch.SCRIPT_FIELDS);
+
         propertyDescriptors = Collections.unmodifiableList(descriptors);
     }
 
diff --git a/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/test/java/org/apache/nifi/processors/elasticsearch/ConsumeElasticsearchTest.java b/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/test/java/org/apache/nifi/processors/elasticsearch/ConsumeElasticsearchTest.java
index 57004bd0d7..d5bd84c891 100644
--- a/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/test/java/org/apache/nifi/processors/elasticsearch/ConsumeElasticsearchTest.java
+++ b/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/test/java/org/apache/nifi/processors/elasticsearch/ConsumeElasticsearchTest.java
@@ -175,7 +175,7 @@ public class ConsumeElasticsearchTest extends SearchElasticsearchTest {
     @Test
     void testNoSorts() throws IOException {
         final TestRunner runner = createRunner(false);
-        runner.removeProperty(ElasticsearchRestProcessor.SORT);
+        runner.removeProperty(ConsumeElasticsearch.SORT);
 
         final Map<String, Object> query = new HashMap<>();
         ((ConsumeElasticsearch) runner.getProcessor()).addSortClause(query, null, runner.getProcessContext());
@@ -186,7 +186,7 @@ public class ConsumeElasticsearchTest extends SearchElasticsearchTest {
     @Test
     void testSingleAdditionalSort() throws IOException {
         final TestRunner runner = createRunner(false);
-        runner.setProperty(ElasticsearchRestProcessor.SORT, "{\"foo\":\"bar\"}");
+        runner.setProperty(ConsumeElasticsearch.SORT, "{\"foo\":\"bar\"}");
 
         final Map<String, Object> query = new HashMap<>();
         ((ConsumeElasticsearch) runner.getProcessor()).addSortClause(query, null, runner.getProcessContext());
@@ -200,7 +200,7 @@ public class ConsumeElasticsearchTest extends SearchElasticsearchTest {
     @Test
     void testMultipleAdditionalSorts() throws IOException {
         final TestRunner runner = createRunner(false);
-        runner.setProperty(ElasticsearchRestProcessor.SORT, "[{\"foo\":\"bar\"},{\"baz\":\"biz\"}]");
+        runner.setProperty(ConsumeElasticsearch.SORT, "[{\"foo\":\"bar\"},{\"baz\":\"biz\"}]");
 
         final Map<String, Object> query = new HashMap<>();
         ((ConsumeElasticsearch) runner.getProcessor()).addSortClause(query, null, runner.getProcessContext());
@@ -216,7 +216,7 @@ public class ConsumeElasticsearchTest extends SearchElasticsearchTest {
     void testTrackingFieldSortAlreadyPresent() throws IOException {
         final TestRunner runner = createRunner(false);
         final String existingRangeFieldSort = String.format("{\"%s\":\"bar\"}", RANGE_FIELD_NAME);
-        runner.setProperty(ElasticsearchRestProcessor.SORT, existingRangeFieldSort);
+        runner.setProperty(ConsumeElasticsearch.SORT, existingRangeFieldSort);
 
         final Map<String, Object> query = new HashMap<>();
         ((ConsumeElasticsearch) runner.getProcessor()).addSortClause(query, null, runner.getProcessContext());
diff --git a/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-test-utils/src/main/java/org/apache/nifi/elasticsearch/integration/AbstractElasticsearchITBase.java b/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-test-utils/src/main/java/org/apache/nifi/elasticsearch/integration/AbstractElasticsearchITBase.java
index f5b8b8e1a4..fef6804e5a 100644
--- a/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-test-utils/src/main/java/org/apache/nifi/elasticsearch/integration/AbstractElasticsearchITBase.java
+++ b/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-test-utils/src/main/java/org/apache/nifi/elasticsearch/integration/AbstractElasticsearchITBase.java
@@ -52,7 +52,7 @@ import static org.apache.http.auth.AuthScope.ANY;
 public abstract class AbstractElasticsearchITBase {
     // default Elasticsearch version should (ideally) match that in the nifi-elasticsearch-bundle#pom.xml for the integration-tests profile
     protected static final DockerImageName IMAGE = DockerImageName
-            .parse(System.getProperty("elasticsearch.docker.image", "docker.elastic.co/elasticsearch/elasticsearch:8.10.2"));
+            .parse(System.getProperty("elasticsearch.docker.image", "docker.elastic.co/elasticsearch/elasticsearch:8.10.3"));
     protected static final String ELASTIC_USER_PASSWORD = System.getProperty("elasticsearch.elastic_user.password", RandomStringUtils.randomAlphanumeric(10, 20));
     private static final int PORT = 9200;
     protected static final ElasticsearchContainer ELASTICSEARCH_CONTAINER = new ElasticsearchContainer(IMAGE)
diff --git a/nifi-nar-bundles/nifi-elasticsearch-bundle/pom.xml b/nifi-nar-bundles/nifi-elasticsearch-bundle/pom.xml
index 97007da5db..809cc44c31 100644
--- a/nifi-nar-bundles/nifi-elasticsearch-bundle/pom.xml
+++ b/nifi-nar-bundles/nifi-elasticsearch-bundle/pom.xml
@@ -94,7 +94,7 @@ language governing permissions and limitations under the License. -->
             </activation>
             <properties>
                 <!-- also update the default Elasticsearch version in nifi-elasticsearch-test-utils#src/main/java/org/apache/nifi/elasticsearch/integration/AbstractElasticsearchITBase.java-->
-                <elasticsearch_docker_image>8.10.2</elasticsearch_docker_image>
+                <elasticsearch_docker_image>8.10.3</elasticsearch_docker_image>
                 <elasticsearch.elastic.password>s3cret</elasticsearch.elastic.password>
             </properties>
             <build>
@@ -125,7 +125,7 @@ language governing permissions and limitations under the License. -->
         <profile>
             <id>elasticsearch7</id>
             <properties>
-                <elasticsearch_docker_image>7.17.13</elasticsearch_docker_image>
+                <elasticsearch_docker_image>7.17.14</elasticsearch_docker_image>
             </properties>
         </profile>
     </profiles>


[nifi] 03/03: NIFI-12211: This closes #7868. add Validator.VALID to username/password on TinkerpopClientService

Posted by jo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

joewitt pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi.git

commit 1a0f509ffc4a2d0b9620d1d322807d3588deb6e4
Author: levilentz <le...@gmail.com>
AuthorDate: Wed Oct 11 10:36:44 2023 -0700

    NIFI-12211: This closes #7868. add Validator.VALID to username/password on TinkerpopClientService
    
    Signed-off-by: Joseph Witt <jo...@apache.org>
---
 .../src/main/java/org/apache/nifi/graph/TinkerpopClientService.java     | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/nifi-nar-bundles/nifi-graph-bundle/nifi-other-graph-services/src/main/java/org/apache/nifi/graph/TinkerpopClientService.java b/nifi-nar-bundles/nifi-graph-bundle/nifi-other-graph-services/src/main/java/org/apache/nifi/graph/TinkerpopClientService.java
index f94a5ba0c5..ac01bb7a3a 100644
--- a/nifi-nar-bundles/nifi-graph-bundle/nifi-other-graph-services/src/main/java/org/apache/nifi/graph/TinkerpopClientService.java
+++ b/nifi-nar-bundles/nifi-graph-bundle/nifi-other-graph-services/src/main/java/org/apache/nifi/graph/TinkerpopClientService.java
@@ -167,6 +167,7 @@ public class TinkerpopClientService extends AbstractControllerService implements
                     " Note: when using a remote.yaml file, this username value (if set) will overload any " +
                     "username set in the YAML file.")
             .required(false)
+            .addValidator(Validator.VALID)
             .expressionLanguageSupported(ExpressionLanguageScope.ENVIRONMENT)
             .build();
 
@@ -178,6 +179,7 @@ public class TinkerpopClientService extends AbstractControllerService implements
                     "password set in the YAML file")
             .required(false)
             .sensitive(true)
+            .addValidator(Validator.VALID)
             .build();
 
     public static final PropertyDescriptor EXTRA_RESOURCE = new PropertyDescriptor.Builder()