You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by GitBox <gi...@apache.org> on 2021/03/24 20:28:12 UTC

[GitHub] [nifi] mattyb149 opened a new pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

mattyb149 opened a new pull request #4933:
URL: https://github.com/apache/nifi/pull/4933


   Thank you for submitting a contribution to Apache NiFi.
   
   Please provide a short description of the PR here:
   
   #### Description of PR
   
   Copies the code from nifi-minifi master branch to a module of nifi. Bumps NiFi dependency version to latest (1.14.0-SNAPSHOT), replaces some MiNiFi components with their NiFi counterparts, moved MiNiFi "common" libraries back into the individual NARs rather than dumping them all into lib/. The size is still large (188MB), a follow-up PR could do some additional refactor to reduce the overall size of the MiNiFi binary. This PR is to produce a functioning MiNiFi binary from inside the NiFi codebase.
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [x] Is there a JIRA ticket associated with this PR? Is it referenced 
        in the commit message?
   
   - [x] Does your PR title start with **NIFI-XXXX** where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
   
   - [x] Has your PR been rebased against the latest commit within the target branch (typically `main`)?
   
   - [ ] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._
   
   ### For code changes:
   - [ ] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder?
   - [x] Have you written or updated unit tests to verify your changes?
   - [x] Have you verified that the full build is successful on JDK 8?
   - [ ] Have you verified that the full build is successful on JDK 11?
   - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? 
   - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`?
   - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`?
   - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] mattyb149 commented on pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
mattyb149 commented on pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#issuecomment-810334432


   I put the C2 stuff back in, all checks passed so hopefully it's in good shape now


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] ottobackwards commented on pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
ottobackwards commented on pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#issuecomment-808464722


   You should throw that in confluence maybe


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] markap14 commented on pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
markap14 commented on pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#issuecomment-812546886


   @mattyb149 I think there are some problems with the LICENSE/NOTICE files for minifi-server-nar and minifi-framework-nar.
   The ones for minifi-framework-nar have almost nothing in them, but the framework nar has tons of dependencies. On the other hand, the minifi-server-nar LICENSE/NOTICE are huge, but the minifi-server-nar only has a few dependencies. Guessing maybe that was done and then a bunch of dependencies moved around? Need to resolve those.
   
   I also noticed that in the minifi-framework-nar, the files are added to src/main/resources whereas everywhere else they are in src/main/resources/META-INF.
   
   Also noticed that the copyrights indicate "Copyright 2014-2018 The Apache Software Foundation" so that should be updated to be 2014-2021. And minifi-framework-nar's NOTICE file also shows a notice that applies to Apache NiFi. Don't think that's really necessary at this point, given that this is part of the Apache NiFi codebase.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] kevdoran edited a comment on pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
kevdoran edited a comment on pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#issuecomment-808411977


   @ottobackwards I wholeheartedly agree. Here is a quick and dirty summary of the current state of things:
   
   | Method of deploying flow | nifi-minifi (java) | nifi-minifi-cpp | 
   | ---|---|---|
   | manually author config.yml | ✔ | ✔ | 
   | nifi template xml -> minifi-toolkit -> config.yml | ✔ | ❌ |
   | change ingestor | ✔ [multiple impls](https://github.com/apache/nifi-minifi/tree/main/minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/configuration/ingestors) | ❌ |
   | [C2 Protocol](https://cwiki.apache.org/confluence/display/MINIFI/C2+Design) | ❌ | ✔ |
   
   C2 Server Implementations that exist in Apache (i.e., not including any community or vendor solutions):
   - [Server for PullHttpChangeIngestor](https://github.com/apache/nifi-minifi/tree/main/minifi-c2)
   - [WIP PoC server-side implementation](https://github.com/apache/nifi-minifi/tree/minifi-c2-server/minifi-c2) of the [C2 Protocol](https://cwiki.apache.org/confluence/display/MINIFI/C2+Design)
   
   @arpadboda - did I miss anything?
   
   Looking towards the future, it is my hope that we can standardize on one approach for both minifi and minifi-cpp. Ideally that is the C2 Protocol with a shared/common server that is compatible with nifi, minifi, and minifi-cpp. I think this PR is the first step towards achieving something like this, as it allows us to much more easily maintain minifi (java) in lockstep with with nifi. From here we have a good foundation to implement a C2 Protocol Client to the same spec supported by minifi-cpp and a corresponding server implementation that can share API code with the java client.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] kevdoran commented on a change in pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
kevdoran commented on a change in pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#discussion_r621611605



##########
File path: minifi/pom.xml
##########
@@ -0,0 +1,1100 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements.  See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+    <parent>
+        <groupId>org.apache.nifi</groupId>
+        <artifactId>nifi</artifactId>
+        <version>1.14.0-SNAPSHOT</version>
+    </parent>
+
+    <groupId>org.apache.nifi.minifi</groupId>
+    <artifactId>minifi</artifactId>
+    <version>1.14.0-SNAPSHOT</version>
+    <packaging>pom</packaging>
+    <description>Apache NiFi - MiNiFi is a child project effort to aid in the collection and transmission of data close to where it is created.</description>
+
+    <modules>
+        <module>minifi-commons</module>
+        <module>minifi-bootstrap</module>
+        <module>minifi-nar-bundles</module>
+        <module>minifi-docs</module>
+        <module>minifi-assembly</module>
+        <module>minifi-toolkit</module>
+        <module>minifi-docker</module>
+        <module>minifi-c2</module>
+        <module>minifi-integration-tests</module>
+    </modules>
+
+    <url>https://nifi.apache.org/minifi</url>
+    <organization>
+        <name>Apache NiFi Project - MiNiFi subproject</name>
+        <url>https://nifi.apache.org/</url>
+    </organization>
+    <licenses>
+        <license>
+            <name>Apache License, Version 2.0</name>
+            <url>https://www.apache.org/licenses/LICENSE-2.0</url>
+        </license>
+    </licenses>
+    <mailingLists>
+        <mailingList>
+            <name>Dev</name>
+            <subscribe>dev-subscribe@nifi.apache.org</subscribe>
+            <unsubscribe>dev-unsubscribe@nifi.apache.org</unsubscribe>
+            <post>dev@nifi.apache.org</post>
+            <archive>https://mail-archives.apache.org/mod_mbox/nifi-dev</archive>
+        </mailingList>
+        <mailingList>
+            <name>Users</name>
+            <subscribe>users-subscribe@nifi.apache.org</subscribe>
+            <unsubscribe>users-unsubscribe@nifi.apache.org</unsubscribe>
+            <post>users@nifi.apache.org</post>
+            <archive>https://mail-archives.apache.org/mod_mbox/nifi-users</archive>
+        </mailingList>
+        <mailingList>
+            <name>Commits</name>
+            <subscribe>commits-subscribe@nifi.apache.org</subscribe>
+            <unsubscribe>commits-unsubscribe@nifi.apache.org</unsubscribe>
+            <post>commits@nifi.apache.org</post>
+            <archive>https://mail-archives.apache.org/mod_mbox/nifi-commits</archive>
+        </mailingList>
+    </mailingLists>
+    <prerequisites>
+        <maven>${maven.min-version}</maven>
+    </prerequisites>
+    <scm>
+        <connection>scm:git:git://git.apache.org/nifi-minifi.git</connection>
+        <developerConnection>scm:git:https://gitbox.apache.org/repos/asf/nifi-minifi.git</developerConnection>
+        <url>https://gitbox.apache.org/repos/asf?p=nifi-minifi.git</url>
+      <tag>HEAD</tag>
+    </scm>

Review comment:
       This can be done after merging this PR, but these repo URLs should be updated :) 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] kevdoran commented on pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
kevdoran commented on pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#issuecomment-808411977


   @ottobackwards I wholeheartedly agree. Here is a quick and dirty summary of the current state of things:
   
   | Method of deploying flow | nifi-minifi (java) | nifi-minifi-cpp | 
   | ---|---|---|
   | manually author config.yml | ✔ | ✔ | 
   | nifi template xml -> minifi-toolkit -> config.yml | ✔ | ❌ |
   | change ingestor | ✔ [multiple impls](https://github.com/apache/nifi-minifi/tree/main/minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/configuration/ingestors) | ❌ |
   | [C2 Protocol](https://cwiki.apache.org/confluence/display/MINIFI/C2+Design) | ❌ | ✔ |
   
   C2 Server Implementations that exist in Apache (i.e., not including any community or vendor solutions):
   - [Server for PullHttpChangeIngestor](https://github.com/apache/nifi-minifi/tree/main/minifi-c2)
   - [WIP PoC server-side implementation](https://github.com/apache/nifi-minifi/tree/minifi-c2-server/minifi-c2) of the [C2 Protocol](https://cwiki.apache.org/confluence/display/MINIFI/C2+Design)
   
   @arpadboda - did I miss anything?
   
   Looking towards the future, it is my hope that we can standardize on one approach for both minifi and minifi-cpp. Ideally that is the C2 Protocol with a shared/common server that is compatible with nifi, minifi, and minifi-cpp.
   
   I think this PR is the first step towards achieving something like this, as it allows us to much more easily maintain minifi (java) in lockstep with with nifi. From here we have a good foundation to implement a C2 Protocol Client to the same spec supported by minifi-cpp and a corresponding server implementation that can share API code with the java client.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] joewitt commented on a change in pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
joewitt commented on a change in pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#discussion_r600862132



##########
File path: minifi/NOTICE
##########
@@ -0,0 +1,7 @@
+Apache NiFi - MiNiFi

Review comment:
       this file should go away completely / This is about the source NOTICE and thus whatever is in here that isn't in the NiFI/NOTICE should get added there and this file removed.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] kevdoran edited a comment on pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
kevdoran edited a comment on pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#issuecomment-808391298


   @gruselglatz - There is no reason this updated minifi coming from the nifi code base should not be compatible with the minifi-c2 server. I took a closer look, and now realize I was mistaking this implementation, which acts as a server for the PullHttpChangeIngestor, with [the implementation on this branch](https://github.com/apache/nifi-minifi/tree/minifi-c2-server/minifi-c2), which was a WIP to update the server based on the aforementioned [C2 Protocol](https://cwiki.apache.org/confluence/display/MINIFI/C2+Design).
   
   Given that this updated minifi retains the Change Investor capabilities, I'm all for keeping the corresponding server, which I agree is useful. (Note - I'm still not sure how actively maintained/patched that server is, so users should be aware of that.)
   
   In the future, I would be in favor of adding a Java client implementation of the [C2 Protocol]((https://cwiki.apache.org/confluence/display/MINIFI/C2+Design)) in this project to be shared be nifi and minifi, which is compatible with the implememtation in `nifi-minifi-cpp`. We should also add a new C2 server implementation for this protocol which could be shared by nifi, minifi, and minifi-cpp, to align all these different flow runtime engines with the same C2 capabilities.
   
   That should not be a consideration for this PR, merely something we could build upon this updated minifi. @mattyb149 - sorry for the back and forth on this, but if the plan is to sunset the nifi-minifi repository, then I do think we should retain that minifi-c2 module here, or else find another home for it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] ottobackwards commented on pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
ottobackwards commented on pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#issuecomment-808395966


   It would be nice if there was some write up about the state of the C2 across the projects so that this was more easily understood and tracked.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] mattyb149 commented on a change in pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
mattyb149 commented on a change in pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#discussion_r600918317



##########
File path: minifi/README.md
##########
@@ -0,0 +1,163 @@
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one or more
+  contributor license agreements.  See the NOTICE file distributed with
+  this work for additional information regarding copyright ownership.
+  The ASF licenses this file to You under the Apache License, Version 2.0
+  (the "License"); you may not use this file except in compliance with
+  the License.  You may obtain a copy of the License at
+      https://www.apache.org/licenses/LICENSE-2.0
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+-->
+# Apache NiFi -  MiNiFi [![Build Status](https://travis-ci.org/apache/nifi-minifi.svg?branch=master)](https://travis-ci.org/apache/nifi-minifi)

Review comment:
       Most of NiFi's README applies to MiNiFi as well (you still have to build the whole thing, plus mailing lists and such are the same), so I just created a section in NiFi's README for the MiNiFi subproject that refers to the binaries, Docker images, etc.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] kevdoran edited a comment on pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
kevdoran edited a comment on pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#issuecomment-808391298


   @gruselglatz - There is no reason this updated minifi coming from the nifi code base should not be compatible with the minifi-c2 server. I took a closer look, and now realize I was mistaking this implementation, which acts as a server for the PullHttpChangeIngestor, with [the implementation on this branch](https://github.com/apache/nifi-minifi/tree/minifi-c2-server/minifi-c2), which was a WIP to update the server based on the aforementioned [C2 Protocol](https://cwiki.apache.org/confluence/display/MINIFI/C2+Design).
   
   Given that this updated minifi retains the Change Investor capabilities, I'm all for keeping the corresponding server, which I agree is useful. (Note - I'm still not sure how actively maintained/patched that server is, so users should be aware of that.)
   
   In the future, I would be in favor of adding a Java client implementation of the [C2 Protocol](https://cwiki.apache.org/confluence/display/MINIFI/C2+Design) in this project to be shared be nifi and minifi, which is compatible with the implememtation in `nifi-minifi-cpp`. We should also add a new C2 server implementation for this protocol which could be shared by nifi, minifi, and minifi-cpp, to align all these different flow runtime engines with the same C2 capabilities.
   
   That should not be a consideration for this PR, merely something we could build upon this updated minifi. @mattyb149 - sorry for the back and forth on this, but if the plan is to sunset the nifi-minifi repository, then I do think we should retain that minifi-c2 module here, or else find another home for it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] phrocker edited a comment on pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
phrocker edited a comment on pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#issuecomment-810993766


   > 
   > 
   > @ottobackwards I wholeheartedly agree. Here is a quick and dirty summary of the current state of things:
   > Method of deploying flow 	nifi-minifi (java) 	nifi-minifi-cpp
   > manually author config.yml 	✔ 	✔
   > nifi template xml -> minifi-toolkit -> config.yml 	✔ 	❌
   > change ingestor 	✔ [multiple impls](https://github.com/apache/nifi-minifi/tree/main/minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/configuration/ingestors) 	❌
   > [C2 Protocol](https://cwiki.apache.org/confluence/display/MINIFI/C2+Design) 	❌ 	✔
   > 
   > C2 Server Implementations that exist in Apache (i.e., not including any community or vendor solutions):
   > 
   >     * [Server for PullHttpChangeIngestor](https://github.com/apache/nifi-minifi/tree/main/minifi-c2)
   > 
   >     * [WIP PoC server-side implementation](https://github.com/apache/nifi-minifi/tree/minifi-c2-server/minifi-c2) of the [C2 Protocol](https://cwiki.apache.org/confluence/display/MINIFI/C2+Design)
   > 
   > 
   > @arpadboda - did I miss anything?
   > 
   > Looking towards the future, it is my hope that we can standardize on one approach for both minifi and minifi-cpp. Ideally that is the C2 Protocol with a shared/common server that is compatible with nifi, minifi, and minifi-cpp. I think this PR is the first step towards achieving something like this, as it allows us to much more easily maintain minifi (java) in lockstep with with nifi. From here we have a good foundation to implement a C2 Protocol Client to the same spec supported by minifi-cpp and a corresponding server implementation that can share API code with the java client.
   
   Sorry for closing. I meant to cancel my reply and I spilled coffee and closed it! I was going to cancel a reply saying I look forward to helping on the nifi-minifi side, because I figured actually helping is more important than commenting...but alas I pulled a fast one on myself. Many apologies for the closure @mattyb149  ! 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] joewitt commented on a change in pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
joewitt commented on a change in pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#discussion_r600861574



##########
File path: minifi/KEYS
##########
@@ -0,0 +1,404 @@
+This file contains the PGP keys of various developers.

Review comment:
       this file should go away completely.  The KEYS file of the NiFi dir itself is sufficient.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] joewitt commented on a change in pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
joewitt commented on a change in pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#discussion_r600863121



##########
File path: minifi/README.md
##########
@@ -0,0 +1,163 @@
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one or more
+  contributor license agreements.  See the NOTICE file distributed with
+  this work for additional information regarding copyright ownership.
+  The ASF licenses this file to You under the Apache License, Version 2.0
+  (the "License"); you may not use this file except in compliance with
+  the License.  You may obtain a copy of the License at
+      https://www.apache.org/licenses/LICENSE-2.0
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+-->
+# Apache NiFi -  MiNiFi [![Build Status](https://travis-ci.org/apache/nifi-minifi.svg?branch=master)](https://travis-ci.org/apache/nifi-minifi)

Review comment:
       seems to me this file should go away and the readme of nifi should be updated to reflect that one might want to create/use the convenience binary of minifi instead of nifi. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] kevdoran commented on pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
kevdoran commented on pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#issuecomment-806173518


   Nice, @mattyb149! This is cool, can't wait to give it a try. 
   
   One quick observation - I would remove the `minifi-c2` submodule. That was a PoC based on https://cwiki.apache.org/confluence/display/MINIFI/C2+Design. Ultimately, the C2 Design was adopted in the nifi-minifi-cpp project, but I think it has diverged from the last time this server effort was worked on, and it is now out of date. I don't think we want to incur the maintenance overhead of adding it to the nifi project repo at this time. Eventually, I would like to revive that effort for a C2 server implementation. We should discuss more the right way to go about that, and we can always bring this over later if this is the right home for it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] phrocker commented on pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
phrocker commented on pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#issuecomment-810993766


   > 
   > 
   > @ottobackwards I wholeheartedly agree. Here is a quick and dirty summary of the current state of things:
   > Method of deploying flow 	nifi-minifi (java) 	nifi-minifi-cpp
   > manually author config.yml 	✔ 	✔
   > nifi template xml -> minifi-toolkit -> config.yml 	✔ 	❌
   > change ingestor 	✔ [multiple impls](https://github.com/apache/nifi-minifi/tree/main/minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/configuration/ingestors) 	❌
   > [C2 Protocol](https://cwiki.apache.org/confluence/display/MINIFI/C2+Design) 	❌ 	✔
   > 
   > C2 Server Implementations that exist in Apache (i.e., not including any community or vendor solutions):
   > 
   >     * [Server for PullHttpChangeIngestor](https://github.com/apache/nifi-minifi/tree/main/minifi-c2)
   > 
   >     * [WIP PoC server-side implementation](https://github.com/apache/nifi-minifi/tree/minifi-c2-server/minifi-c2) of the [C2 Protocol](https://cwiki.apache.org/confluence/display/MINIFI/C2+Design)
   > 
   > 
   > @arpadboda - did I miss anything?
   > 
   > Looking towards the future, it is my hope that we can standardize on one approach for both minifi and minifi-cpp. Ideally that is the C2 Protocol with a shared/common server that is compatible with nifi, minifi, and minifi-cpp. I think this PR is the first step towards achieving something like this, as it allows us to much more easily maintain minifi (java) in lockstep with with nifi. From here we have a good foundation to implement a C2 Protocol Client to the same spec supported by minifi-cpp and a corresponding server implementation that can share API code with the java client.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] kevdoran commented on pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
kevdoran commented on pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#issuecomment-808391298


   @gruselglatz - There is no reason this updated minifi coming from the nifi code base should not be compatible with the minifi-c2 server. I took a closer look, and now realize I was mistaking this implementation, which acts as a server for the PullHttpChangeIngestor, with [the implementation on this branch](https://github.com/apache/nifi-minifi/tree/minifi-c2-server/minifi-c2), which was a WIP to update the server based on the aforementioned [C2 Protocol](https://cwiki.apache.org/confluence/display/MINIFI/C2+Design).
   
   Given that this updated minifi retains the Change Investor capabilities, I'm all for keeping the corresponding server, which I agree is useful. 
   
   In the future, I would be in favor of adding a Java client implementation of the [C2 Protocol] in this project to be shared be nifi and minifi, which is compatible with the implememtation in `nifi-minifi-cpp`. We should also add a new C2 server implementation for this protocol which could be shared by nifi, minifi, and minifi-cpp, to align all these different flow runtime engines with the same C2 capabilities.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] kevdoran commented on pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
kevdoran commented on pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#issuecomment-811098431


   @phrocker better to accidentally close rather than accidentally merge haha 😆 
   
   @mattyb149 this looks good, I'll be putting it through the paces this week trying it out!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] phrocker edited a comment on pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
phrocker edited a comment on pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#issuecomment-810993766


   > 
   > 
   > @ottobackwards I wholeheartedly agree. Here is a quick and dirty summary of the current state of things:
   > Method of deploying flow 	nifi-minifi (java) 	nifi-minifi-cpp
   > manually author config.yml 	✔ 	✔
   > nifi template xml -> minifi-toolkit -> config.yml 	✔ 	❌
   > change ingestor 	✔ [multiple impls](https://github.com/apache/nifi-minifi/tree/main/minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/configuration/ingestors) 	❌
   > [C2 Protocol](https://cwiki.apache.org/confluence/display/MINIFI/C2+Design) 	❌ 	✔
   > 
   > C2 Server Implementations that exist in Apache (i.e., not including any community or vendor solutions):
   > 
   >     * [Server for PullHttpChangeIngestor](https://github.com/apache/nifi-minifi/tree/main/minifi-c2)
   > 
   >     * [WIP PoC server-side implementation](https://github.com/apache/nifi-minifi/tree/minifi-c2-server/minifi-c2) of the [C2 Protocol](https://cwiki.apache.org/confluence/display/MINIFI/C2+Design)
   > 
   > 
   > @arpadboda - did I miss anything?
   > 
   > Looking towards the future, it is my hope that we can standardize on one approach for both minifi and minifi-cpp. Ideally that is the C2 Protocol with a shared/common server that is compatible with nifi, minifi, and minifi-cpp. I think this PR is the first step towards achieving something like this, as it allows us to much more easily maintain minifi (java) in lockstep with with nifi. From here we have a good foundation to implement a C2 Protocol Client to the same spec supported by minifi-cpp and a corresponding server implementation that can share API code with the java client.
   
   Sorry for closing. I meant to cancel my reply and I spilled coffee and closed it! I was going to cancel a reply saying I look forward to help on the nifi-minifi side ( I figured actually helping is more important than responding ). 
   
   Apologies for the closure @mattyb149  ! 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] kevdoran edited a comment on pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
kevdoran edited a comment on pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#issuecomment-808391298


   @gruselglatz - There is no reason this updated minifi coming from the nifi code base should not be compatible with the minifi-c2 server. I took a closer look, and now realize I was mistaking this implementation, which acts as a server for the PullHttpChangeIngestor, with [the implementation on this branch](https://github.com/apache/nifi-minifi/tree/minifi-c2-server/minifi-c2), which was a WIP to update the server based on the aforementioned [C2 Protocol](https://cwiki.apache.org/confluence/display/MINIFI/C2+Design).
   
   Given that this updated minifi retains the Change Ingestor capabilities, I'm all for keeping the corresponding server, which I agree is useful. (Note - I'm still not sure how actively maintained/patched that server is, so users should be aware of that.)
   
   In the future, I would be in favor of adding a Java client implementation of the [C2 Protocol](https://cwiki.apache.org/confluence/display/MINIFI/C2+Design) in this project to be shared be nifi and minifi, which is compatible with the implememtation in `nifi-minifi-cpp`. We should also add a new C2 server implementation for this protocol which could be shared by nifi, minifi, and minifi-cpp, to align all these different flow runtime engines with the same C2 capabilities.
   
   That should not be a consideration for this PR, merely something we could build upon this updated minifi. @mattyb149 - sorry for the back and forth on this, but if the plan is to sunset the nifi-minifi repository, then I do think we should retain that minifi-c2 module here, or else find another home for it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] ottobackwards commented on pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
ottobackwards commented on pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#issuecomment-808478506


   @joewitt @kevdoran 
   
   I sincerely apologize.  I totally understand.  Think of it more coming from a place of "I don't know enough about this core area to feel comfortable doing it my self" than "I couldn't be bothered to do this, you should do it".
   
   That being said, I'll add a page in confluence, as long as we are OK with me just cut and pasting
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] ottobackwards commented on pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
ottobackwards commented on pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#issuecomment-808481897


   https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=177048430&flashId=624428108
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] gruselglatz commented on pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
gruselglatz commented on pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#issuecomment-806401588


   > 
   > 
   > Nice, @mattyb149! This is cool, can't wait to give it a try.
   > 
   > One quick observation - I would remove the `minifi-c2` submodule. That was a PoC based on https://cwiki.apache.org/confluence/display/MINIFI/C2+Design. Ultimately, the C2 Design was adopted in the nifi-minifi-cpp project, but I think it has diverged from the last time this server effort was worked on, and it is now out of date. I don't think we want to incur the maintenance overhead of adding it to the nifi project repo at this time. Eventually, I would like to revive that effort for a C2 server implementation. We should discuss more the right way to go about that, and we can always bring this over later if this is the right home for it.
   
   Hey @kevdoran , we are heavily using Minifi-C2 in its current state, and are pretty satisfied by its current feature-set, at the same time we are very excited about the update for Minifi. But will this mean that C2 wont be compatible anymore? Or can we still use it, so our minifies can pull their config from it? Or should or can we switch to nifi registry for that? Or does it simply mean that the minifi-cpp c2 server will be compatible to minifi-java? (Sry if this is not the right place to discuss this?)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] kevdoran closed pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
kevdoran closed pull request #4933:
URL: https://github.com/apache/nifi/pull/4933


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] joewitt commented on a change in pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
joewitt commented on a change in pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#discussion_r600861959



##########
File path: minifi/LICENSE
##########
@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004

Review comment:
       this file should go away completely / This is about the source LICENSE and thus whatever is in here that isn't in the NiFI/LICENSE should get added there and this file removed.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] phrocker closed pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
phrocker closed pull request #4933:
URL: https://github.com/apache/nifi/pull/4933


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [nifi] joewitt commented on pull request #4933: MINIFI-422: Integrate MiNiFi Java codebase into NiFi

Posted by GitBox <gi...@apache.org>.
joewitt commented on pull request #4933:
URL: https://github.com/apache/nifi/pull/4933#issuecomment-808476116


   @ottobackwards "It would be nice if there was some write up " .... "You should throw that in confluence maybe".
   
   I trust your intent is good but these comments are not helpful.  None of us really need a lot of mentorship in this regard.  What we need is help *doing* the things.  So you think something could benefit from another unit test?  Write it.  You think something would help grow community and engagement?  Write it.  Not a single one of us is strapped for ideas.  The limiting factor now is execution bandwidth.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org