You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Robert Cohen <ro...@anu.edu.au.INVALID> on 2024/12/12 01:09:43 UTC

importing code into minifi fron nifi

Im having trouble getting code into minifi 2 from nifi 2.

The documentation here 
https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md

says

# Export the dataflow in JSON format.
# Move your new .json file to |minifi/conf|.
# Rename your .json file /flow.json.raw/.


Im using the option in nifi to "download flow definition"

However when I load that into minifi, I get an error message.

java.lang.Exception: Unable to load flow due to: 
org.apache.nifi.controller.serialization.FlowSerializationException: 
Could not parse flow as a VersionedDataflow


When I compare the file to the dummy flow file that ships with minifi.

The file that ships with minifi starts with

{
   "encodingVersion": {
     "majorVersion": 2,
     "minorVersion": 0
   },


But the export from nifi has

{"flowContents":{"identifier":"a2ac574c-635d-3df5-baf0-6cbf4dd5bdec",


So the 2 files dont appear to be the same format.

Is there some other method of exporting code from nifi that Im missing?

Re: importing code into minifi fron nifi

Posted by Kedvessy Péter <pk...@gmail.com>.
Hi,

I have addressed the invalid flow json issue:
https://issues.apache.org/jira/browse/NIFI-14149

Regards,
Peter

Kedvessy Péter <pk...@gmail.com> ezt írta (időpont: 2025. jan. 10., P,
10:49):

> Hi Robert,
>
> Have you checked your minifi agent logs? Specifically the minifi-app.log.
> There should be an error describing the reason for the failed
> connection/communication with the remote NiFi instance... Is
> https://odc-nifi-test.nci.org.au:444/nifi the right URI of your remote
> NiFi instance where you are expecting flow files from the minifi agent?
>
> Regards,
> Peter
>
> Robert Cohen <ro...@anu.edu.au.invalid> ezt írta (időpont: 2025.
> jan. 8., Sze, 5:20):
>
>> OK, I fixed the validation. I had an error in my bootstrap.conf.
>>
>> Its not ideal that the toolkit transform-nifi generates a invalid flow
>> file.
>>
>>
>> Now I can get minifi to start. But the code isnt working.
>>
>> This is a trivial "ping" . It just uses a GenFlowFile to make a flow.
>> And sends it to a remote process group. The remote process group lives
>> in another nifi instance, and just logs and discards the flows.
>>
>>
>> I had to move the code in the source nifi into sub group. And I exported
>> that. And ran it through the converter.
>>
>> Inside nifi, its working fine.
>>
>> But in the minifi, its generating the flows. I can see them piling up in
>> the WAL. But they arent being transmitted. Theres nothing being received
>> by the remote nifi.
>>
>> OK, I fixed the validation. I had an error in my bootstrap.conf.
>>
>> Its not ideal that the toolkit transform-nifi generates a invalid flow
>> file.
>>
>>
>> Now I can get minifi to start. But the code isnt working.
>>
>> This is a trivial "ping" . It just uses a GenFlowFile to make a flow.
>> And sends it to a remote process group. The remote process group lives
>> in another nifi instance, and just logs and discards the flows.
>>
>>
>> I had to move the code in the source nifi into sub group. And I exported
>> that. And ran it through the converter.
>>
>> Inside nifi, its working fine.
>>
>> But in the minifi, its generating the flows. I can see them piling up in
>> the WAL. But they arent being transmitted. Theres nothing being received
>> by the remote nifi.
>>
>>
>>
>> On 8/1/2025 2:33 pm, Robert Cohen wrote:
>> > I googled the error message.
>> >
>> > I found a thread here which seems the same
>> >
>> >
>> https://community.cloudera.com/t5/Support-Questions/Unable-to-start-MiNiFi-because-of-an-error-caused-due-to-the/m-p/393219
>> >
>> >
>> >
>> > One thing it suggests is editing the flow file and changing the
>> > following setting from 0.
>> >
>> > |"maxTimerDrivenThreadCount":5 But when I do that minifi fails to
>> > start as it says the flow file fails validation Does the flow file
>> > contain a checksum? |
>> >
>> >
>> > On 8/1/2025 1:27 pm, Robert Cohen wrote:
>> >> Thanks for that. It clears some things up.
>> >>
>> >> Im not sure if the doco is supposed to be suitable for nifi 1/2 or
>> >> just nifi 2.
>> >>
>> >> But you might want to clarify in the doco whether config files are
>> >> just used by nifi 1. Just used by nifi 2. Or supported in nifi 2 for
>> >> legacy. But not recommended.
>> >>
>> >>
>> >> Anyway, Ive put my code from a nifi 2.1.0 into minifi 2.1.0.
>> >>
>> >> I get the following error
>> >>
>> >> 2025-01-08 13:16:09,065 ERROR [MiNiFi logging handler]
>> >> org.apache.nifi.minifi.StdErr Failed to start flow service: Unable to
>> >> load flow due to:
>> >> org.apache.nifi.controller.serialization.FlowSynchronizationException:
>> >> java.lang.IllegalArgumentException: Cannot set max number of threads
>> >> to less than 1
>> >>
>> >> I attach the converted flow file
>> >>
>> >> I can't see any obvious parameter related to max number of threads.
>> >>
>> >>
>> >>
>> >> On 7/1/2025 6:56 pm, Kedvessy Péter wrote:
>> >>> [You don't often get email from pkedvessy@gmail.com. Learn why this
>> >>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>> >>>
>> >>> Hi Robert,
>> >>>
>> >>> Thanks for reporting the documentation issues. I filed a JIRA to fix
>> >>> them:
>> >>> https://issues.apache.org/jira/browse/NIFI-14093
>> >>>
>> >>>
>> >>> In terms of security configuration, you can follow the following
>> guide:
>> >>>
>> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/System_Admin_Guide.md#security-properties-in-bootstrapconf
>> >>>
>> >>>
>> >>> About the different configuration files you mentioned:
>> >>> - bootstrap.conf is the main configuration file what we should update
>> >>> - minifi.properties is a generated properties file based on the
>> >>> bootstrap.conf
>> >>> - nifi.properties is the legacy configuration file of minifi
>> >>> - config.yaml is the legacy flow configuration, now we have the json
>> >>> one
>> >>> what you have tried to import previously
>> >>>
>> >>> I hope it helps to set up your minfi agent.
>> >>>
>> >>> Regards,
>> >>> Peter
>> >>>
>> >>> Robert Cohen <ro...@anu.edu.au.invalid> ezt írta (időpont:
>> 2025.
>> >>> jan. 7., K, 6:19):
>> >>>
>> >>>> I'm having a certain amount of trouble getting good info on
>> >>>> administration of minifi 2.
>> >>>>
>> >>>> The README file says thats documentation can be found at
>> >>>>
>> >>>> https://nifi.apache.org/minifi
>> >>>> and
>> >>>> https://cwiki.apache.org/confluence/display/NIFI/MiNiFi
>> >>>>
>> >>>>
>> >>>> neither of which URL works.
>> >>>>
>> >>>>
>> >>>> Im assuming the real documentation is at
>> >>>>
>> >>>>
>> >>>>
>> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/System_Admin_Guide.md#config-file
>> >>>>
>> >>>>
>> >>>>
>> >>>> But Im having some Im finding it a bit confusing.
>> >>>>
>> >>>> I want to configure the keystore/truststore and sensitive prop
>> >>>> passwords.
>> >>>>
>> >>>> It appears I can configure them either in bootstrap.conf or
>> >>>> minifi.properties. Is either to be prefered over the other?
>> >>>>
>> >>>> The documention mentions nifi.properties and config.yml. But doesnt
>> >>>> mention minifi.properties.
>> >>>>
>> >>>> Im assuming the mentions of config.yml are leftover from minifi 1
>> >>>> documentation. Or does minifi 2 use it for backwards compatability.
>> >>>>
>> >>>> And I assume nifi.properties has been renamed to minifi.properties
>> and
>> >>>> the documentation hasnt caught up.
>> >>>>
>> >>>>
>> >>>> Actually dumping strings out of the jar files, I can find instances
>> of
>> >>>> nifi.properties. But I cant find any references to minifi.properties.
>> >>>>
>> >>>> However the tarball contains a minifi.properties file but not a
>> >>>> nifi.properties file.
>> >>>>
>> >>>> So Im really not sure what is going on.
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> On 6/1/2025 4:40 pm, Robert Cohen wrote:
>> >>>>> Ahh, it appears the functionality is there. Its just that the README
>> >>>>> file hasnt been updated,
>> >>>>>
>> >>>>> Theres now a transform-nifi command in addition to the transform-yml
>> >>>>> command.
>> >>>>>
>> >>>>>
>> >>>>> On 6/1/2025 2:59 pm, Robert Cohen wrote:
>> >>>>>> [You don't often get email from robert.cohen@anu.edu.au.invalid.
>> >>>>>> Learn why this is important at
>> >>>>>> https://aka.ms/LearnAboutSenderIdentification ]
>> >>>>>>
>> >>>>>> I had a look at the minifi toolkit.
>> >>>>>>
>> >>>>>> However, it only appears to contain functionality for converting
>> >>>>>> legacy
>> >>>>>> minifi config to minifi 2 config
>> >>>>>>
>> >>>>>> Heres an excerpt from  the minifi-tookit 2.1.0 README file
>> >>>>>>
>> >>>>>>        Valid commands include:
>> >>>>>>        transform-yml: Transforms legacy MiNiFi flow config YAML
>> into
>> >>>>>> MiNiFi flow config JSON
>> >>>>>>
>> >>>>>> ## Example
>> >>>>>> - You have an older version of MiNiFi located in
>> >>>>>> <legacy_minifi_directory>.
>> >>>>>> - You would like upgrade to the latest version of MiNiFi. You
>> >>>>>> downloaded
>> >>>>>> and extracted the latest MiNiFi into <latest_minifi_directory>.
>> >>>>>> - Run the following command to migrate the flow and the bootstrap
>> >>>>>> config
>> >>>>>> ```
>> >>>>>> ./config.sh transform-yml <legacy_minifi_directory>/conf/config.yml
>> >>>>>> <legacy_minifi_directory>/conf/bootstrap.conf
>> >>>>>> <latest_minifi_directory>/conf/flow.json.raw
>> >>>>>> <latest_minifi_directory>/conf/bootstrap.conf
>> >>>>>> ```
>> >>>>>>
>> >>>>>>
>> >>>>>> Is there functionality somewhere to convert a nifi 2.x flow file
>> >>>>>> to a
>> >>>>>> minifi 2 flow file,
>> >>>>>>
>> >>>>>>
>> >>>>>> On 12/12/2024 6:48 pm, Kedvessy Péter wrote:
>> >>>>>>> [You don't often get email from pkedvessy@gmail.com. Learn why
>> this
>> >>>>>>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>> >>>>>>>
>> >>>>>>> Hi Robert,
>> >>>>>>>
>> >>>>>>> Seems like the documentation has not been updated. I'm going to
>> >>>>>>> file
>> >>>>>>> a JIRA
>> >>>>>>> ticket to apply doc update about how we can import NiFi 2
>> >>>>>>> exported flow
>> >>>>>>> files into MiNiFi.
>> >>>>>>> In a nutshell: before moving the json file into minifi, we should
>> >>>>>>> convert
>> >>>>>>> it with minifi toolkit. Related ticket for more details:
>> >>>>>>> https://issues.apache.org/jira/browse/NIFI-12644
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Regards,
>> >>>>>>> Peter
>> >>>>>>>
>> >>>>>>> Robert Cohen <ro...@anu.edu.au.invalid> ezt írta (időpont:
>> >>>> 2024.
>> >>>>>>> dec. 12., Cs, 2:11):
>> >>>>>>>
>> >>>>>>>> Im having trouble getting code into minifi 2 from nifi 2.
>> >>>>>>>>
>> >>>>>>>> The documentation here
>> >>>>>>>>
>> >>>>>>>>
>> >>>>
>> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md
>> >>>>
>> >>>>>>>>
>> >>>>>>>> says
>> >>>>>>>>
>> >>>>>>>> # Export the dataflow in JSON format.
>> >>>>>>>> # Move your new .json file to |minifi/conf|.
>> >>>>>>>> # Rename your .json file /flow.json.raw/.
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> Im using the option in nifi to "download flow definition"
>> >>>>>>>>
>> >>>>>>>> However when I load that into minifi, I get an error message.
>> >>>>>>>>
>> >>>>>>>> java.lang.Exception: Unable to load flow due to:
>> >>>>>>>>
>> org.apache.nifi.controller.serialization.FlowSerializationException:
>> >>>>>>>>
>> >>>>>>>> Could not parse flow as a VersionedDataflow
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> When I compare the file to the dummy flow file that ships with
>> >>>>>>>> minifi.
>> >>>>>>>>
>> >>>>>>>> The file that ships with minifi starts with
>> >>>>>>>>
>> >>>>>>>> {
>> >>>>>>>>      "encodingVersion": {
>> >>>>>>>>        "majorVersion": 2,
>> >>>>>>>>        "minorVersion": 0
>> >>>>>>>>      },
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> But the export from nifi has
>> >>>>>>>>
>> >>>>>>>>
>> {"flowContents":{"identifier":"a2ac574c-635d-3df5-baf0-6cbf4dd5bdec",
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> So the 2 files dont appear to be the same format.
>> >>>>>>>>
>> >>>>>>>> Is there some other method of exporting code from nifi that Im
>> >>>>>>>> missing?
>> >>>>>>>>
>> >>>>>>>>
>>
>

Re: importing code into minifi fron nifi

Posted by Robert Cohen <ro...@anu.edu.au.INVALID>.
I had a quick look at what the minifi toolkit is doing.

It converts Enabled -> Running for Processors. But not RemoteProcessGroups.


Something that surprises me is the fact that nifi config and minifi 
config are different at all.

They are both compiled from the same code base. So I would have thought 
the config parsing would be shared.

They made the effort to change from yaml to json config for minifi 2.

Why didnt they just make nifi and minifi config identical.

Then there would be no need for  a converter at all. Just export it from 
nifi and shove it into minifi.

Minifi could ignore the GUI related attributes.



On 20/1/2025 5:25 pm, Robert Cohen wrote:
> Actually, I suspect the export from nifi is supposed to be 
> scheduledState: Enabled.
>
> I think its probably the job of the minifi-toolkit to change that from 
> Enabled to Running.
>
>

Re: importing code into minifi fron nifi

Posted by Robert Cohen <ro...@anu.edu.au.INVALID>.
Actually, I suspect the export from nifi is supposed to be 
scheduledState: Enabled.

I think its probably the job of the minifi-toolkit to change that from 
Enabled to Running.



On 20/1/2025 5:07 pm, Robert Cohen wrote:
> OK, I believe Ive spotted the problem with exporting code from nifi 
> 2.1.0 to minifi 2.1.0
>
> I have 2 minifi 2.1.0 flow.json.raw files.
>
> The first came from nifi 2.1.0 and converted with minifi-toolkikt 
> transform-nifi.
>
> The second came from minifi 1.28.1 converted with minifi-toolkit 
> transform-yml
>
>
> The code that came from minifi works. The code that comes from nifi 
> doesnt.
>
> Comparing the two, I spotted a difference.
>
>
> For the remote process groups, in the working code, the input ports 
> have "scheduledState": "RUNNING",
>
> For the non working code that came from nifi, the remote process 
> groups have input ports with "scheduledState": "ENABLED",
>
>
> If I take the nifi output and change the scheduledState to Running, 
> the code then works.
>
> I checked the exported file coming from nifi, to see if the problem 
> was with the nifi export or in the converter.
>
> The problem is there in the exported file. So it appears to be a nifi 
> problem, not a toolkit problem.
>
>

Re: importing code into minifi fron nifi

Posted by Robert Cohen <ro...@anu.edu.au.INVALID>.
OK, I believe Ive spotted the problem with exporting code from nifi 
2.1.0 to minifi 2.1.0

I have 2 minifi 2.1.0 flow.json.raw files.

The first came from nifi 2.1.0 and converted with minifi-toolkikt 
transform-nifi.

The second came from minifi 1.28.1 converted with minifi-toolkit 
transform-yml


The code that came from minifi works. The code that comes from nifi doesnt.

Comparing the two, I spotted a difference.


For the remote process groups, in the working code, the input ports have 
"scheduledState": "RUNNING",

For the non working code that came from nifi, the remote process groups 
have input ports with "scheduledState": "ENABLED",


If I take the nifi output and change the scheduledState to Running, the 
code then works.

I checked the exported file coming from nifi, to see if the problem was 
with the nifi export or in the converter.

The problem is there in the exported file. So it appears to be a nifi 
problem, not a toolkit problem.



On 16/1/2025 5:19 pm, Robert Cohen wrote:
> Ive tried running the transform-yml. The output appears to be minifi 
> json. Not nifi json.
>
> So I think that doco is misleading.
>
>
> The code exported as a template from niifi 1.28.1, converted to a 
> minifi 1.28.1 yml then converted to a minifi 2.1.0 json  appears to work.
>
> Wheres the same code exported from nifi 2.1.0 and converted to minifi 
> json doesnt.
>
> I'll start comparing the two and see if I can spot the problem.
>
>
>
> On 16/1/2025 4:00 pm, Robert Cohen wrote:
>>
>> Im coming at it a different way. Im trying to take the code from 
>> minifi 1.28.1 and convert it to minifi 2.1.0.
>>
>>
>> Now the minifi toolkit 2.1.0 help says
>>
>> Valid commands include:
>> transform-nifi: Transform NiFi flow JSON format into MiNifi flow JSON 
>> format
>> transform-yml: Transform MiNiFi config YAML into NiFi flow JSON format
>>
>> So does that mean I need to use transform-yml to convert it to nifi 
>> flow format.
>>
>> Then use transform-nifi to convert it from nifi flow format to minifi 
>> flow format
>>
>>
>> On 14/1/2025 9:25 pm, Robert Cohen wrote:
>>> No, the TLS keystore and truststores are all set up.
>>> And everything works fine in the original nifi sender.
>>> Its only when I copy the code from the original nifi sending host to 
>>> either a minifi or another nifi, that things stop working.
>>>
>>> On 14/01/2025 5:07 pm, Matthew Hawkins wrote:
>>>> [You don't often get email from hawko2600@gmail.com. Learn why this 
>>>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>>>>
>>>> Hi Robert,
>>>>
>>>> You mentioned you have not setup the TLS side. From about Nifi 1.17 
>>>> or so,
>>>> this has been the default and expected behavior.
>>>>
>>>> Please setup the Nifi instance properly with TLS certs (they can be 
>>>> self
>>>> signed, I have a script on GitHub gist to help with this previously 
>>>> shared
>>>> with this forum).
>>>>
>>>> You drop the other side's client certificate into trusted keystore and
>>>> setup an association via the SSLContextService associated with the 
>>>> RPG.
>>>>
>>>> There's not a lot of logging available when this fails, but having 
>>>> done it
>>>> several times with third party REST interconnects and Nifi to Nifi 
>>>> I can
>>>> assure you there's something in the code since back then that just 
>>>> won't
>>>> let unencrypted unauthenticated sessions transmit. As a security
>>>> professional, I like it fails (but dislike it fails silently or 
>>>> weirdly).
>>>>
>>>> Kind regards,
>>>>
>>>> On Tue, 14 Jan 2025, 16:23 Robert Cohen, 
>>>> <ro...@anu.edu.au.invalid>
>>>> wrote:
>>>>
>>>>> Ive got no idea whats going on with this thing.
>>>>>
>>>>> I thought it might have been a minifi issue or a converter issue, 
>>>>> so Im
>>>>> trying to export a process group  from my source  nifi and push it 
>>>>> into
>>>>> another standalone nifi.
>>>>>
>>>>> Since I dont care about the GUI, I havent bothered to set up
>>>>> authentication or authorization.
>>>>>
>>>>> I just took the the exported flow definition and renamed it to
>>>>> flow.json.raw
>>>>>
>>>>> But the code doesnt appear to be running properly in the 
>>>>> standalone nifi
>>>>> either.
>>>>>
>>>>> The app log doesnt show anything.  Just the regular checkpoints.
>>>>>
>>>>> None of the processors or the RPG appear to be running.
>>>>>
>>>>> I'll have to try and get the GUI working so I can log into it and see
>>>>> whats going on.
>>>>>
>>>>>
>>>>>
>>>>> On 13/1/2025 1:21 pm, Robert Cohen wrote:
>>>>>> Yes, Ive checked the logs.
>>>>>>
>>>>>> There are no error messages of any kind or any indication that its
>>>>>> attempting to transmit anything at all.
>>>>>>
>>>>>> Its as if the remote process group is either disconnected or 
>>>>>> disabled.
>>>>>>
>>>>>>
>>>>>> The nifi URL works when its in the originating source nifi. And Im
>>>>>> running the minifi on the same host as the source nifi. So things 
>>>>>> like
>>>>>> firewalls/routing arent an issue.
>>>>>>
>>>>>> And https:odc-nifi-test.nci.org.au:444/nifi is the correct URL 
>>>>>> for the
>>>>>> destination remote NIFI.
>>>>>>
>>>>>> Im going to try a flow file converted from the minifi 1 
>>>>>> config.ynl, If
>>>>>> that works I;ll have sonething to compare.
>>>>>>
>>>>>> I'll also try importing the exported flow into a new NIFI and see if
>>>>>> theres something wrong with the exported file.
>>>>>>
>>>>>>
>>>>>> On 10/1/2025 8:49 pm, Kedvessy Péter wrote:
>>>>>>> [You don't often get email frompkedvessy@gmail.com. Learn why 
>>>>>>> this is
>>>>>>> important athttps://aka.ms/LearnAboutSenderIdentification ]
>>>>>>>
>>>>>>> Hi Robert,
>>>>>>>
>>>>>>> Have you checked your minifi agent logs? Specifically the
>>>>>>> minifi-app.log.
>>>>>>> There should be an error describing the reason for the failed
>>>>>>> connection/communication with the remote NiFi instance... Is
>>>>>>> https:odc-nifi-test.nci.org.au:444/nifi the right URI of your 
>>>>>>> remote
>>>>>>> NiFi
>>>>>>> instance where you are expecting flow files from the minifi agent?
>>>>>>>
>>>>>>> Regards,
>>>>>>> Peter
>>>>>>>
>>>

Re: importing code into minifi fron nifi

Posted by Martin Zink <ma...@apache.org>.
Hey Robert, If everything else fails, you might want to try out
https://github.com/adamdebreceni/c2-server (note: I'm a contributor to
that project) it's a new open source (apache 2 licensed) C2 server
that helps with design, deployment and monitoring via webui for both
minifi java and minifi c++.

On Thu, Jan 16, 2025 at 7:20 AM Robert Cohen
<ro...@anu.edu.au.invalid> wrote:
>
> Ive tried running the transform-yml. The output appears to be minifi
> json. Not nifi json.
>
> So I think that doco is misleading.
>
>
> The code exported as a template from niifi 1.28.1, converted to a minifi
> 1.28.1 yml then converted to a minifi 2.1.0 json  appears to work.
>
> Wheres the same code exported from nifi 2.1.0 and converted to minifi
> json doesnt.
>
> I'll start comparing the two and see if I can spot the problem.
>
>
>
> On 16/1/2025 4:00 pm, Robert Cohen wrote:
> >
> > Im coming at it a different way. Im trying to take the code from
> > minifi 1.28.1 and convert it to minifi 2.1.0.
> >
> >
> > Now the minifi toolkit 2.1.0 help says
> >
> > Valid commands include:
> > transform-nifi: Transform NiFi flow JSON format into MiNifi flow JSON
> > format
> > transform-yml: Transform MiNiFi config YAML into NiFi flow JSON format
> >
> > So does that mean I need to use transform-yml to convert it to nifi
> > flow format.
> >
> > Then use transform-nifi to convert it from nifi flow format to minifi
> > flow format
> >
> >
> > On 14/1/2025 9:25 pm, Robert Cohen wrote:
> >> No, the TLS keystore and truststores are all set up.
> >> And everything works fine in the original nifi sender.
> >> Its only when I copy the code from the original nifi sending host to
> >> either a minifi or another nifi, that things stop working.
> >>
> >> On 14/01/2025 5:07 pm, Matthew Hawkins wrote:
> >>> [You don't often get email from hawko2600@gmail.com. Learn why this
> >>> is important at https://aka.ms/LearnAboutSenderIdentification ]
> >>>
> >>> Hi Robert,
> >>>
> >>> You mentioned you have not setup the TLS side. From about Nifi 1.17
> >>> or so,
> >>> this has been the default and expected behavior.
> >>>
> >>> Please setup the Nifi instance properly with TLS certs (they can be
> >>> self
> >>> signed, I have a script on GitHub gist to help with this previously
> >>> shared
> >>> with this forum).
> >>>
> >>> You drop the other side's client certificate into trusted keystore and
> >>> setup an association via the SSLContextService associated with the RPG.
> >>>
> >>> There's not a lot of logging available when this fails, but having
> >>> done it
> >>> several times with third party REST interconnects and Nifi to Nifi I
> >>> can
> >>> assure you there's something in the code since back then that just
> >>> won't
> >>> let unencrypted unauthenticated sessions transmit. As a security
> >>> professional, I like it fails (but dislike it fails silently or
> >>> weirdly).
> >>>
> >>> Kind regards,
> >>>
> >>> On Tue, 14 Jan 2025, 16:23 Robert Cohen,
> >>> <ro...@anu.edu.au.invalid>
> >>> wrote:
> >>>
> >>>> Ive got no idea whats going on with this thing.
> >>>>
> >>>> I thought it might have been a minifi issue or a converter issue,
> >>>> so Im
> >>>> trying to export a process group  from my source  nifi and push it
> >>>> into
> >>>> another standalone nifi.
> >>>>
> >>>> Since I dont care about the GUI, I havent bothered to set up
> >>>> authentication or authorization.
> >>>>
> >>>> I just took the the exported flow definition and renamed it to
> >>>> flow.json.raw
> >>>>
> >>>> But the code doesnt appear to be running properly in the standalone
> >>>> nifi
> >>>> either.
> >>>>
> >>>> The app log doesnt show anything.  Just the regular checkpoints.
> >>>>
> >>>> None of the processors or the RPG appear to be running.
> >>>>
> >>>> I'll have to try and get the GUI working so I can log into it and see
> >>>> whats going on.
> >>>>
> >>>>
> >>>>
> >>>> On 13/1/2025 1:21 pm, Robert Cohen wrote:
> >>>>> Yes, Ive checked the logs.
> >>>>>
> >>>>> There are no error messages of any kind or any indication that its
> >>>>> attempting to transmit anything at all.
> >>>>>
> >>>>> Its as if the remote process group is either disconnected or
> >>>>> disabled.
> >>>>>
> >>>>>
> >>>>> The nifi URL works when its in the originating source nifi. And Im
> >>>>> running the minifi on the same host as the source nifi. So things
> >>>>> like
> >>>>> firewalls/routing arent an issue.
> >>>>>
> >>>>> And https:odc-nifi-test.nci.org.au:444/nifi is the correct URL for
> >>>>> the
> >>>>> destination remote NIFI.
> >>>>>
> >>>>> Im going to try a flow file converted from the minifi 1
> >>>>> config.ynl, If
> >>>>> that works I;ll have sonething to compare.
> >>>>>
> >>>>> I'll also try importing the exported flow into a new NIFI and see if
> >>>>> theres something wrong with the exported file.
> >>>>>
> >>>>>
> >>>>> On 10/1/2025 8:49 pm, Kedvessy Péter wrote:
> >>>>>> [You don't often get email frompkedvessy@gmail.com. Learn why
> >>>>>> this is
> >>>>>> important athttps://aka.ms/LearnAboutSenderIdentification ]
> >>>>>>
> >>>>>> Hi Robert,
> >>>>>>
> >>>>>> Have you checked your minifi agent logs? Specifically the
> >>>>>> minifi-app.log.
> >>>>>> There should be an error describing the reason for the failed
> >>>>>> connection/communication with the remote NiFi instance... Is
> >>>>>> https:odc-nifi-test.nci.org.au:444/nifi the right URI of your remote
> >>>>>> NiFi
> >>>>>> instance where you are expecting flow files from the minifi agent?
> >>>>>>
> >>>>>> Regards,
> >>>>>> Peter
> >>>>>>
> >>

Re: importing code into minifi fron nifi

Posted by Robert Cohen <ro...@anu.edu.au.INVALID>.
Ive tried running the transform-yml. The output appears to be minifi 
json. Not nifi json.

So I think that doco is misleading.


The code exported as a template from niifi 1.28.1, converted to a minifi 
1.28.1 yml then converted to a minifi 2.1.0 json  appears to work.

Wheres the same code exported from nifi 2.1.0 and converted to minifi 
json doesnt.

I'll start comparing the two and see if I can spot the problem.



On 16/1/2025 4:00 pm, Robert Cohen wrote:
>
> Im coming at it a different way. Im trying to take the code from 
> minifi 1.28.1 and convert it to minifi 2.1.0.
>
>
> Now the minifi toolkit 2.1.0 help says
>
> Valid commands include:
> transform-nifi: Transform NiFi flow JSON format into MiNifi flow JSON 
> format
> transform-yml: Transform MiNiFi config YAML into NiFi flow JSON format
>
> So does that mean I need to use transform-yml to convert it to nifi 
> flow format.
>
> Then use transform-nifi to convert it from nifi flow format to minifi 
> flow format
>
>
> On 14/1/2025 9:25 pm, Robert Cohen wrote:
>> No, the TLS keystore and truststores are all set up.
>> And everything works fine in the original nifi sender.
>> Its only when I copy the code from the original nifi sending host to 
>> either a minifi or another nifi, that things stop working.
>>
>> On 14/01/2025 5:07 pm, Matthew Hawkins wrote:
>>> [You don't often get email from hawko2600@gmail.com. Learn why this 
>>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>>>
>>> Hi Robert,
>>>
>>> You mentioned you have not setup the TLS side. From about Nifi 1.17 
>>> or so,
>>> this has been the default and expected behavior.
>>>
>>> Please setup the Nifi instance properly with TLS certs (they can be 
>>> self
>>> signed, I have a script on GitHub gist to help with this previously 
>>> shared
>>> with this forum).
>>>
>>> You drop the other side's client certificate into trusted keystore and
>>> setup an association via the SSLContextService associated with the RPG.
>>>
>>> There's not a lot of logging available when this fails, but having 
>>> done it
>>> several times with third party REST interconnects and Nifi to Nifi I 
>>> can
>>> assure you there's something in the code since back then that just 
>>> won't
>>> let unencrypted unauthenticated sessions transmit. As a security
>>> professional, I like it fails (but dislike it fails silently or 
>>> weirdly).
>>>
>>> Kind regards,
>>>
>>> On Tue, 14 Jan 2025, 16:23 Robert Cohen, 
>>> <ro...@anu.edu.au.invalid>
>>> wrote:
>>>
>>>> Ive got no idea whats going on with this thing.
>>>>
>>>> I thought it might have been a minifi issue or a converter issue, 
>>>> so Im
>>>> trying to export a process group  from my source  nifi and push it 
>>>> into
>>>> another standalone nifi.
>>>>
>>>> Since I dont care about the GUI, I havent bothered to set up
>>>> authentication or authorization.
>>>>
>>>> I just took the the exported flow definition and renamed it to
>>>> flow.json.raw
>>>>
>>>> But the code doesnt appear to be running properly in the standalone 
>>>> nifi
>>>> either.
>>>>
>>>> The app log doesnt show anything.  Just the regular checkpoints.
>>>>
>>>> None of the processors or the RPG appear to be running.
>>>>
>>>> I'll have to try and get the GUI working so I can log into it and see
>>>> whats going on.
>>>>
>>>>
>>>>
>>>> On 13/1/2025 1:21 pm, Robert Cohen wrote:
>>>>> Yes, Ive checked the logs.
>>>>>
>>>>> There are no error messages of any kind or any indication that its
>>>>> attempting to transmit anything at all.
>>>>>
>>>>> Its as if the remote process group is either disconnected or 
>>>>> disabled.
>>>>>
>>>>>
>>>>> The nifi URL works when its in the originating source nifi. And Im
>>>>> running the minifi on the same host as the source nifi. So things 
>>>>> like
>>>>> firewalls/routing arent an issue.
>>>>>
>>>>> And https:odc-nifi-test.nci.org.au:444/nifi is the correct URL for 
>>>>> the
>>>>> destination remote NIFI.
>>>>>
>>>>> Im going to try a flow file converted from the minifi 1 
>>>>> config.ynl, If
>>>>> that works I;ll have sonething to compare.
>>>>>
>>>>> I'll also try importing the exported flow into a new NIFI and see if
>>>>> theres something wrong with the exported file.
>>>>>
>>>>>
>>>>> On 10/1/2025 8:49 pm, Kedvessy Péter wrote:
>>>>>> [You don't often get email frompkedvessy@gmail.com. Learn why 
>>>>>> this is
>>>>>> important athttps://aka.ms/LearnAboutSenderIdentification ]
>>>>>>
>>>>>> Hi Robert,
>>>>>>
>>>>>> Have you checked your minifi agent logs? Specifically the
>>>>>> minifi-app.log.
>>>>>> There should be an error describing the reason for the failed
>>>>>> connection/communication with the remote NiFi instance... Is
>>>>>> https:odc-nifi-test.nci.org.au:444/nifi the right URI of your remote
>>>>>> NiFi
>>>>>> instance where you are expecting flow files from the minifi agent?
>>>>>>
>>>>>> Regards,
>>>>>> Peter
>>>>>>
>>

Re: importing code into minifi fron nifi

Posted by Robert Cohen <ro...@anu.edu.au.INVALID>.
So Ive confirned that transferring code from nifi 2.1.0 to minifi 2.1.0 
works at some level.

I created an even simpler flow. It just has GenFlowFile -> 
LogAttributes. And the LogAttributes terminates the flow.

I was able to export that and run it through the converter and it works.


So my problem must be something to do with the RemoteProcessGroup.


Im coming at it a different way. Im trying to take the code from minifi 
1.28.1 and convert it to minifi 2.1.0.


Now the minifi toolkit 2.1.0 help says

Valid commands include:
transform-nifi: Transform NiFi flow JSON format into MiNifi flow JSON format
transform-yml: Transform MiNiFi config YAML into NiFi flow JSON format

So does that mean I need to use transform-yml to convert it to nifi flow 
format.

Then use transform-nifi to convert it from nifi flow format to minifi 
flow format


On 14/1/2025 9:25 pm, Robert Cohen wrote:
> No, the TLS keystore and truststores are all set up.
> And everything works fine in the original nifi sender.
> Its only when I copy the code from the original nifi sending host to 
> either a minifi or another nifi, that things stop working.
>
> On 14/01/2025 5:07 pm, Matthew Hawkins wrote:
>> [You don't often get email from hawko2600@gmail.com. Learn why this 
>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>>
>> Hi Robert,
>>
>> You mentioned you have not setup the TLS side. From about Nifi 1.17 
>> or so,
>> this has been the default and expected behavior.
>>
>> Please setup the Nifi instance properly with TLS certs (they can be self
>> signed, I have a script on GitHub gist to help with this previously 
>> shared
>> with this forum).
>>
>> You drop the other side's client certificate into trusted keystore and
>> setup an association via the SSLContextService associated with the RPG.
>>
>> There's not a lot of logging available when this fails, but having 
>> done it
>> several times with third party REST interconnects and Nifi to Nifi I can
>> assure you there's something in the code since back then that just won't
>> let unencrypted unauthenticated sessions transmit. As a security
>> professional, I like it fails (but dislike it fails silently or 
>> weirdly).
>>
>> Kind regards,
>>
>> On Tue, 14 Jan 2025, 16:23 Robert Cohen, 
>> <ro...@anu.edu.au.invalid>
>> wrote:
>>
>>> Ive got no idea whats going on with this thing.
>>>
>>> I thought it might have been a minifi issue or a converter issue, so Im
>>> trying to export a process group  from my source  nifi and push it into
>>> another standalone nifi.
>>>
>>> Since I dont care about the GUI, I havent bothered to set up
>>> authentication or authorization.
>>>
>>> I just took the the exported flow definition and renamed it to
>>> flow.json.raw
>>>
>>> But the code doesnt appear to be running properly in the standalone 
>>> nifi
>>> either.
>>>
>>> The app log doesnt show anything.  Just the regular checkpoints.
>>>
>>> None of the processors or the RPG appear to be running.
>>>
>>> I'll have to try and get the GUI working so I can log into it and see
>>> whats going on.
>>>
>>>
>>>
>>> On 13/1/2025 1:21 pm, Robert Cohen wrote:
>>>> Yes, Ive checked the logs.
>>>>
>>>> There are no error messages of any kind or any indication that its
>>>> attempting to transmit anything at all.
>>>>
>>>> Its as if the remote process group is either disconnected or disabled.
>>>>
>>>>
>>>> The nifi URL works when its in the originating source nifi. And Im
>>>> running the minifi on the same host as the source nifi. So things like
>>>> firewalls/routing arent an issue.
>>>>
>>>> And https:odc-nifi-test.nci.org.au:444/nifi is the correct URL for the
>>>> destination remote NIFI.
>>>>
>>>> Im going to try a flow file converted from the minifi 1 config.ynl, If
>>>> that works I;ll have sonething to compare.
>>>>
>>>> I'll also try importing the exported flow into a new NIFI and see if
>>>> theres something wrong with the exported file.
>>>>
>>>>
>>>> On 10/1/2025 8:49 pm, Kedvessy Péter wrote:
>>>>> [You don't often get email frompkedvessy@gmail.com. Learn why this is
>>>>> important athttps://aka.ms/LearnAboutSenderIdentification ]
>>>>>
>>>>> Hi Robert,
>>>>>
>>>>> Have you checked your minifi agent logs? Specifically the
>>>>> minifi-app.log.
>>>>> There should be an error describing the reason for the failed
>>>>> connection/communication with the remote NiFi instance... Is
>>>>> https:odc-nifi-test.nci.org.au:444/nifi the right URI of your remote
>>>>> NiFi
>>>>> instance where you are expecting flow files from the minifi agent?
>>>>>
>>>>> Regards,
>>>>> Peter
>>>>>
>

Re: importing code into minifi fron nifi

Posted by Robert Cohen <ro...@anu.edu.au.INVALID>.
No, the TLS keystore and truststores are all set up.
And everything works fine in the original nifi sender.
Its only when I copy the code from the original nifi sending host to 
either a minifi or another nifi, that things stop working.

On 14/01/2025 5:07 pm, Matthew Hawkins wrote:
> [You don't often get email from hawko2600@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
>
> Hi Robert,
>
> You mentioned you have not setup the TLS side. From about Nifi 1.17 or so,
> this has been the default and expected behavior.
>
> Please setup the Nifi instance properly with TLS certs (they can be self
> signed, I have a script on GitHub gist to help with this previously shared
> with this forum).
>
> You drop the other side's client certificate into trusted keystore and
> setup an association via the SSLContextService associated with the RPG.
>
> There's not a lot of logging available when this fails, but having done it
> several times with third party REST interconnects and Nifi to Nifi I can
> assure you there's something in the code since back then that just won't
> let unencrypted unauthenticated sessions transmit. As a security
> professional, I like it fails (but dislike it fails silently or weirdly).
>
> Kind regards,
>
> On Tue, 14 Jan 2025, 16:23 Robert Cohen, <ro...@anu.edu.au.invalid>
> wrote:
>
>> Ive got no idea whats going on with this thing.
>>
>> I thought it might have been a minifi issue or a converter issue, so Im
>> trying to export a process group  from my source  nifi and push it into
>> another standalone nifi.
>>
>> Since I dont care about the GUI, I havent bothered to set up
>> authentication or authorization.
>>
>> I just took the the exported flow definition and renamed it to
>> flow.json.raw
>>
>> But the code doesnt appear to be running properly in the standalone nifi
>> either.
>>
>> The app log doesnt show anything.  Just the regular checkpoints.
>>
>> None of the processors or the RPG appear to be running.
>>
>> I'll have to try and get the GUI working so I can log into it and see
>> whats going on.
>>
>>
>>
>> On 13/1/2025 1:21 pm, Robert Cohen wrote:
>>> Yes, Ive checked the logs.
>>>
>>> There are no error messages of any kind or any indication that its
>>> attempting to transmit anything at all.
>>>
>>> Its as if the remote process group is either disconnected or disabled.
>>>
>>>
>>> The nifi URL works when its in the originating source nifi. And Im
>>> running the minifi on the same host as the source nifi. So things like
>>> firewalls/routing arent an issue.
>>>
>>> And https:odc-nifi-test.nci.org.au:444/nifi is the correct URL for the
>>> destination remote NIFI.
>>>
>>> Im going to try a flow file converted from the minifi 1 config.ynl, If
>>> that works I;ll have sonething to compare.
>>>
>>> I'll also try importing the exported flow into a new NIFI and see if
>>> theres something wrong with the exported file.
>>>
>>>
>>> On 10/1/2025 8:49 pm, Kedvessy Péter wrote:
>>>> [You don't often get email frompkedvessy@gmail.com. Learn why this is
>>>> important athttps://aka.ms/LearnAboutSenderIdentification ]
>>>>
>>>> Hi Robert,
>>>>
>>>> Have you checked your minifi agent logs? Specifically the
>>>> minifi-app.log.
>>>> There should be an error describing the reason for the failed
>>>> connection/communication with the remote NiFi instance... Is
>>>> https:odc-nifi-test.nci.org.au:444/nifi the right URI of your remote
>>>> NiFi
>>>> instance where you are expecting flow files from the minifi agent?
>>>>
>>>> Regards,
>>>> Peter
>>>>

Re: importing code into minifi fron nifi

Posted by Matthew Hawkins <ha...@gmail.com>.
Hi Robert,

You mentioned you have not setup the TLS side. From about Nifi 1.17 or so,
this has been the default and expected behavior.

Please setup the Nifi instance properly with TLS certs (they can be self
signed, I have a script on GitHub gist to help with this previously shared
with this forum).

You drop the other side's client certificate into trusted keystore and
setup an association via the SSLContextService associated with the RPG.

There's not a lot of logging available when this fails, but having done it
several times with third party REST interconnects and Nifi to Nifi I can
assure you there's something in the code since back then that just won't
let unencrypted unauthenticated sessions transmit. As a security
professional, I like it fails (but dislike it fails silently or weirdly).

Kind regards,

On Tue, 14 Jan 2025, 16:23 Robert Cohen, <ro...@anu.edu.au.invalid>
wrote:

> Ive got no idea whats going on with this thing.
>
> I thought it might have been a minifi issue or a converter issue, so Im
> trying to export a process group  from my source  nifi and push it into
> another standalone nifi.
>
> Since I dont care about the GUI, I havent bothered to set up
> authentication or authorization.
>
> I just took the the exported flow definition and renamed it to
> flow.json.raw
>
> But the code doesnt appear to be running properly in the standalone nifi
> either.
>
> The app log doesnt show anything.  Just the regular checkpoints.
>
> None of the processors or the RPG appear to be running.
>
> I'll have to try and get the GUI working so I can log into it and see
> whats going on.
>
>
>
> On 13/1/2025 1:21 pm, Robert Cohen wrote:
> > Yes, Ive checked the logs.
> >
> > There are no error messages of any kind or any indication that its
> > attempting to transmit anything at all.
> >
> > Its as if the remote process group is either disconnected or disabled.
> >
> >
> > The nifi URL works when its in the originating source nifi. And Im
> > running the minifi on the same host as the source nifi. So things like
> > firewalls/routing arent an issue.
> >
> > And https:odc-nifi-test.nci.org.au:444/nifi is the correct URL for the
> > destination remote NIFI.
> >
> > Im going to try a flow file converted from the minifi 1 config.ynl, If
> > that works I;ll have sonething to compare.
> >
> > I'll also try importing the exported flow into a new NIFI and see if
> > theres something wrong with the exported file.
> >
> >
> > On 10/1/2025 8:49 pm, Kedvessy Péter wrote:
> >> [You don't often get email frompkedvessy@gmail.com. Learn why this is
> >> important athttps://aka.ms/LearnAboutSenderIdentification ]
> >>
> >> Hi Robert,
> >>
> >> Have you checked your minifi agent logs? Specifically the
> >> minifi-app.log.
> >> There should be an error describing the reason for the failed
> >> connection/communication with the remote NiFi instance... Is
> >> https:odc-nifi-test.nci.org.au:444/nifi the right URI of your remote
> >> NiFi
> >> instance where you are expecting flow files from the minifi agent?
> >>
> >> Regards,
> >> Peter
> >>
>

Re: importing code into minifi fron nifi

Posted by Robert Cohen <ro...@anu.edu.au.INVALID>.
Ive got no idea whats going on with this thing.

I thought it might have been a minifi issue or a converter issue, so Im 
trying to export a process group  from my source  nifi and push it into 
another standalone nifi.

Since I dont care about the GUI, I havent bothered to set up 
authentication or authorization.

I just took the the exported flow definition and renamed it to flow.json.raw

But the code doesnt appear to be running properly in the standalone nifi 
either.

The app log doesnt show anything.  Just the regular checkpoints.

None of the processors or the RPG appear to be running.

I'll have to try and get the GUI working so I can log into it and see 
whats going on.



On 13/1/2025 1:21 pm, Robert Cohen wrote:
> Yes, Ive checked the logs.
>
> There are no error messages of any kind or any indication that its 
> attempting to transmit anything at all.
>
> Its as if the remote process group is either disconnected or disabled.
>
>
> The nifi URL works when its in the originating source nifi. And Im 
> running the minifi on the same host as the source nifi. So things like 
> firewalls/routing arent an issue.
>
> And https:odc-nifi-test.nci.org.au:444/nifi is the correct URL for the 
> destination remote NIFI.
>
> Im going to try a flow file converted from the minifi 1 config.ynl, If 
> that works I;ll have sonething to compare.
>
> I'll also try importing the exported flow into a new NIFI and see if 
> theres something wrong with the exported file.
>
>
> On 10/1/2025 8:49 pm, Kedvessy Péter wrote:
>> [You don't often get email frompkedvessy@gmail.com. Learn why this is 
>> important athttps://aka.ms/LearnAboutSenderIdentification ]
>>
>> Hi Robert,
>>
>> Have you checked your minifi agent logs? Specifically the 
>> minifi-app.log.
>> There should be an error describing the reason for the failed
>> connection/communication with the remote NiFi instance... Is
>> https:odc-nifi-test.nci.org.au:444/nifi the right URI of your remote 
>> NiFi
>> instance where you are expecting flow files from the minifi agent?
>>
>> Regards,
>> Peter
>>

Re: importing code into minifi fron nifi

Posted by Robert Cohen <ro...@anu.edu.au.INVALID>.
Yes, Ive checked the logs.

There are no error messages of any kind or any indication that its 
attempting to transmit anything at all.

Its as if the remote process group is either disconnected or disabled.


The nifi URL works when its in the originating source nifi. And Im 
running the minifi on the same host as the source nifi. So things like 
firewalls/routing arent an issue.

And https:odc-nifi-test.nci.org.au:444/nifi is the correct URL for the 
destination remote NIFI.

Im going to try a flow file converted from the minifi 1 config.ynl, If 
that works I;ll have sonething to compare.

I'll also try importing the exported flow into a new NIFI and see if 
theres something wrong with the exported file.


On 10/1/2025 8:49 pm, Kedvessy Péter wrote:
> [You don't often get email frompkedvessy@gmail.com. Learn why this is important athttps://aka.ms/LearnAboutSenderIdentification ]
>
> Hi Robert,
>
> Have you checked your minifi agent logs? Specifically the minifi-app.log.
> There should be an error describing the reason for the failed
> connection/communication with the remote NiFi instance... Is
> https:odc-nifi-test.nci.org.au:444/nifi the right URI of your remote NiFi
> instance where you are expecting flow files from the minifi agent?
>
> Regards,
> Peter
>

Re: importing code into minifi fron nifi

Posted by Kedvessy Péter <pk...@gmail.com>.
Hi Robert,

Have you checked your minifi agent logs? Specifically the minifi-app.log.
There should be an error describing the reason for the failed
connection/communication with the remote NiFi instance... Is
https://odc-nifi-test.nci.org.au:444/nifi the right URI of your remote NiFi
instance where you are expecting flow files from the minifi agent?

Regards,
Peter

Robert Cohen <ro...@anu.edu.au.invalid> ezt írta (időpont: 2025.
jan. 8., Sze, 5:20):

> OK, I fixed the validation. I had an error in my bootstrap.conf.
>
> Its not ideal that the toolkit transform-nifi generates a invalid flow
> file.
>
>
> Now I can get minifi to start. But the code isnt working.
>
> This is a trivial "ping" . It just uses a GenFlowFile to make a flow.
> And sends it to a remote process group. The remote process group lives
> in another nifi instance, and just logs and discards the flows.
>
>
> I had to move the code in the source nifi into sub group. And I exported
> that. And ran it through the converter.
>
> Inside nifi, its working fine.
>
> But in the minifi, its generating the flows. I can see them piling up in
> the WAL. But they arent being transmitted. Theres nothing being received
> by the remote nifi.
>
> OK, I fixed the validation. I had an error in my bootstrap.conf.
>
> Its not ideal that the toolkit transform-nifi generates a invalid flow
> file.
>
>
> Now I can get minifi to start. But the code isnt working.
>
> This is a trivial "ping" . It just uses a GenFlowFile to make a flow.
> And sends it to a remote process group. The remote process group lives
> in another nifi instance, and just logs and discards the flows.
>
>
> I had to move the code in the source nifi into sub group. And I exported
> that. And ran it through the converter.
>
> Inside nifi, its working fine.
>
> But in the minifi, its generating the flows. I can see them piling up in
> the WAL. But they arent being transmitted. Theres nothing being received
> by the remote nifi.
>
>
>
> On 8/1/2025 2:33 pm, Robert Cohen wrote:
> > I googled the error message.
> >
> > I found a thread here which seems the same
> >
> >
> https://community.cloudera.com/t5/Support-Questions/Unable-to-start-MiNiFi-because-of-an-error-caused-due-to-the/m-p/393219
> >
> >
> >
> > One thing it suggests is editing the flow file and changing the
> > following setting from 0.
> >
> > |"maxTimerDrivenThreadCount":5 But when I do that minifi fails to
> > start as it says the flow file fails validation Does the flow file
> > contain a checksum? |
> >
> >
> > On 8/1/2025 1:27 pm, Robert Cohen wrote:
> >> Thanks for that. It clears some things up.
> >>
> >> Im not sure if the doco is supposed to be suitable for nifi 1/2 or
> >> just nifi 2.
> >>
> >> But you might want to clarify in the doco whether config files are
> >> just used by nifi 1. Just used by nifi 2. Or supported in nifi 2 for
> >> legacy. But not recommended.
> >>
> >>
> >> Anyway, Ive put my code from a nifi 2.1.0 into minifi 2.1.0.
> >>
> >> I get the following error
> >>
> >> 2025-01-08 13:16:09,065 ERROR [MiNiFi logging handler]
> >> org.apache.nifi.minifi.StdErr Failed to start flow service: Unable to
> >> load flow due to:
> >> org.apache.nifi.controller.serialization.FlowSynchronizationException:
> >> java.lang.IllegalArgumentException: Cannot set max number of threads
> >> to less than 1
> >>
> >> I attach the converted flow file
> >>
> >> I can't see any obvious parameter related to max number of threads.
> >>
> >>
> >>
> >> On 7/1/2025 6:56 pm, Kedvessy Péter wrote:
> >>> [You don't often get email from pkedvessy@gmail.com. Learn why this
> >>> is important at https://aka.ms/LearnAboutSenderIdentification ]
> >>>
> >>> Hi Robert,
> >>>
> >>> Thanks for reporting the documentation issues. I filed a JIRA to fix
> >>> them:
> >>> https://issues.apache.org/jira/browse/NIFI-14093
> >>>
> >>>
> >>> In terms of security configuration, you can follow the following guide:
> >>>
> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/System_Admin_Guide.md#security-properties-in-bootstrapconf
> >>>
> >>>
> >>> About the different configuration files you mentioned:
> >>> - bootstrap.conf is the main configuration file what we should update
> >>> - minifi.properties is a generated properties file based on the
> >>> bootstrap.conf
> >>> - nifi.properties is the legacy configuration file of minifi
> >>> - config.yaml is the legacy flow configuration, now we have the json
> >>> one
> >>> what you have tried to import previously
> >>>
> >>> I hope it helps to set up your minfi agent.
> >>>
> >>> Regards,
> >>> Peter
> >>>
> >>> Robert Cohen <ro...@anu.edu.au.invalid> ezt írta (időpont:
> 2025.
> >>> jan. 7., K, 6:19):
> >>>
> >>>> I'm having a certain amount of trouble getting good info on
> >>>> administration of minifi 2.
> >>>>
> >>>> The README file says thats documentation can be found at
> >>>>
> >>>> https://nifi.apache.org/minifi
> >>>> and
> >>>> https://cwiki.apache.org/confluence/display/NIFI/MiNiFi
> >>>>
> >>>>
> >>>> neither of which URL works.
> >>>>
> >>>>
> >>>> Im assuming the real documentation is at
> >>>>
> >>>>
> >>>>
> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/System_Admin_Guide.md#config-file
> >>>>
> >>>>
> >>>>
> >>>> But Im having some Im finding it a bit confusing.
> >>>>
> >>>> I want to configure the keystore/truststore and sensitive prop
> >>>> passwords.
> >>>>
> >>>> It appears I can configure them either in bootstrap.conf or
> >>>> minifi.properties. Is either to be prefered over the other?
> >>>>
> >>>> The documention mentions nifi.properties and config.yml. But doesnt
> >>>> mention minifi.properties.
> >>>>
> >>>> Im assuming the mentions of config.yml are leftover from minifi 1
> >>>> documentation. Or does minifi 2 use it for backwards compatability.
> >>>>
> >>>> And I assume nifi.properties has been renamed to minifi.properties and
> >>>> the documentation hasnt caught up.
> >>>>
> >>>>
> >>>> Actually dumping strings out of the jar files, I can find instances of
> >>>> nifi.properties. But I cant find any references to minifi.properties.
> >>>>
> >>>> However the tarball contains a minifi.properties file but not a
> >>>> nifi.properties file.
> >>>>
> >>>> So Im really not sure what is going on.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On 6/1/2025 4:40 pm, Robert Cohen wrote:
> >>>>> Ahh, it appears the functionality is there. Its just that the README
> >>>>> file hasnt been updated,
> >>>>>
> >>>>> Theres now a transform-nifi command in addition to the transform-yml
> >>>>> command.
> >>>>>
> >>>>>
> >>>>> On 6/1/2025 2:59 pm, Robert Cohen wrote:
> >>>>>> [You don't often get email from robert.cohen@anu.edu.au.invalid.
> >>>>>> Learn why this is important at
> >>>>>> https://aka.ms/LearnAboutSenderIdentification ]
> >>>>>>
> >>>>>> I had a look at the minifi toolkit.
> >>>>>>
> >>>>>> However, it only appears to contain functionality for converting
> >>>>>> legacy
> >>>>>> minifi config to minifi 2 config
> >>>>>>
> >>>>>> Heres an excerpt from  the minifi-tookit 2.1.0 README file
> >>>>>>
> >>>>>>        Valid commands include:
> >>>>>>        transform-yml: Transforms legacy MiNiFi flow config YAML into
> >>>>>> MiNiFi flow config JSON
> >>>>>>
> >>>>>> ## Example
> >>>>>> - You have an older version of MiNiFi located in
> >>>>>> <legacy_minifi_directory>.
> >>>>>> - You would like upgrade to the latest version of MiNiFi. You
> >>>>>> downloaded
> >>>>>> and extracted the latest MiNiFi into <latest_minifi_directory>.
> >>>>>> - Run the following command to migrate the flow and the bootstrap
> >>>>>> config
> >>>>>> ```
> >>>>>> ./config.sh transform-yml <legacy_minifi_directory>/conf/config.yml
> >>>>>> <legacy_minifi_directory>/conf/bootstrap.conf
> >>>>>> <latest_minifi_directory>/conf/flow.json.raw
> >>>>>> <latest_minifi_directory>/conf/bootstrap.conf
> >>>>>> ```
> >>>>>>
> >>>>>>
> >>>>>> Is there functionality somewhere to convert a nifi 2.x flow file
> >>>>>> to a
> >>>>>> minifi 2 flow file,
> >>>>>>
> >>>>>>
> >>>>>> On 12/12/2024 6:48 pm, Kedvessy Péter wrote:
> >>>>>>> [You don't often get email from pkedvessy@gmail.com. Learn why
> this
> >>>>>>> is important at https://aka.ms/LearnAboutSenderIdentification ]
> >>>>>>>
> >>>>>>> Hi Robert,
> >>>>>>>
> >>>>>>> Seems like the documentation has not been updated. I'm going to
> >>>>>>> file
> >>>>>>> a JIRA
> >>>>>>> ticket to apply doc update about how we can import NiFi 2
> >>>>>>> exported flow
> >>>>>>> files into MiNiFi.
> >>>>>>> In a nutshell: before moving the json file into minifi, we should
> >>>>>>> convert
> >>>>>>> it with minifi toolkit. Related ticket for more details:
> >>>>>>> https://issues.apache.org/jira/browse/NIFI-12644
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Regards,
> >>>>>>> Peter
> >>>>>>>
> >>>>>>> Robert Cohen <ro...@anu.edu.au.invalid> ezt írta (időpont:
> >>>> 2024.
> >>>>>>> dec. 12., Cs, 2:11):
> >>>>>>>
> >>>>>>>> Im having trouble getting code into minifi 2 from nifi 2.
> >>>>>>>>
> >>>>>>>> The documentation here
> >>>>>>>>
> >>>>>>>>
> >>>>
> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md
> >>>>
> >>>>>>>>
> >>>>>>>> says
> >>>>>>>>
> >>>>>>>> # Export the dataflow in JSON format.
> >>>>>>>> # Move your new .json file to |minifi/conf|.
> >>>>>>>> # Rename your .json file /flow.json.raw/.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Im using the option in nifi to "download flow definition"
> >>>>>>>>
> >>>>>>>> However when I load that into minifi, I get an error message.
> >>>>>>>>
> >>>>>>>> java.lang.Exception: Unable to load flow due to:
> >>>>>>>>
> org.apache.nifi.controller.serialization.FlowSerializationException:
> >>>>>>>>
> >>>>>>>> Could not parse flow as a VersionedDataflow
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> When I compare the file to the dummy flow file that ships with
> >>>>>>>> minifi.
> >>>>>>>>
> >>>>>>>> The file that ships with minifi starts with
> >>>>>>>>
> >>>>>>>> {
> >>>>>>>>      "encodingVersion": {
> >>>>>>>>        "majorVersion": 2,
> >>>>>>>>        "minorVersion": 0
> >>>>>>>>      },
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> But the export from nifi has
> >>>>>>>>
> >>>>>>>>
> {"flowContents":{"identifier":"a2ac574c-635d-3df5-baf0-6cbf4dd5bdec",
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> So the 2 files dont appear to be the same format.
> >>>>>>>>
> >>>>>>>> Is there some other method of exporting code from nifi that Im
> >>>>>>>> missing?
> >>>>>>>>
> >>>>>>>>
>

Re: importing code into minifi fron nifi

Posted by Robert Cohen <ro...@anu.edu.au.INVALID>.
OK, I fixed the validation. I had an error in my bootstrap.conf.

Its not ideal that the toolkit transform-nifi generates a invalid flow file.


Now I can get minifi to start. But the code isnt working.

This is a trivial "ping" . It just uses a GenFlowFile to make a flow. 
And sends it to a remote process group. The remote process group lives 
in another nifi instance, and just logs and discards the flows.


I had to move the code in the source nifi into sub group. And I exported 
that. And ran it through the converter.

Inside nifi, its working fine.

But in the minifi, its generating the flows. I can see them piling up in 
the WAL. But they arent being transmitted. Theres nothing being received 
by the remote nifi.

OK, I fixed the validation. I had an error in my bootstrap.conf.

Its not ideal that the toolkit transform-nifi generates a invalid flow file.


Now I can get minifi to start. But the code isnt working.

This is a trivial "ping" . It just uses a GenFlowFile to make a flow. 
And sends it to a remote process group. The remote process group lives 
in another nifi instance, and just logs and discards the flows.


I had to move the code in the source nifi into sub group. And I exported 
that. And ran it through the converter.

Inside nifi, its working fine.

But in the minifi, its generating the flows. I can see them piling up in 
the WAL. But they arent being transmitted. Theres nothing being received 
by the remote nifi.



On 8/1/2025 2:33 pm, Robert Cohen wrote:
> I googled the error message.
>
> I found a thread here which seems the same
>
> https://community.cloudera.com/t5/Support-Questions/Unable-to-start-MiNiFi-because-of-an-error-caused-due-to-the/m-p/393219 
>
>
>
> One thing it suggests is editing the flow file and changing the 
> following setting from 0.
>
> |"maxTimerDrivenThreadCount":5 But when I do that minifi fails to 
> start as it says the flow file fails validation Does the flow file 
> contain a checksum? |
>
>
> On 8/1/2025 1:27 pm, Robert Cohen wrote:
>> Thanks for that. It clears some things up.
>>
>> Im not sure if the doco is supposed to be suitable for nifi 1/2 or 
>> just nifi 2.
>>
>> But you might want to clarify in the doco whether config files are 
>> just used by nifi 1. Just used by nifi 2. Or supported in nifi 2 for 
>> legacy. But not recommended.
>>
>>
>> Anyway, Ive put my code from a nifi 2.1.0 into minifi 2.1.0.
>>
>> I get the following error
>>
>> 2025-01-08 13:16:09,065 ERROR [MiNiFi logging handler] 
>> org.apache.nifi.minifi.StdErr Failed to start flow service: Unable to 
>> load flow due to: 
>> org.apache.nifi.controller.serialization.FlowSynchronizationException: 
>> java.lang.IllegalArgumentException: Cannot set max number of threads 
>> to less than 1
>>
>> I attach the converted flow file
>>
>> I can't see any obvious parameter related to max number of threads.
>>
>>
>>
>> On 7/1/2025 6:56 pm, Kedvessy Péter wrote:
>>> [You don't often get email from pkedvessy@gmail.com. Learn why this 
>>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>>>
>>> Hi Robert,
>>>
>>> Thanks for reporting the documentation issues. I filed a JIRA to fix 
>>> them:
>>> https://issues.apache.org/jira/browse/NIFI-14093 
>>>
>>>
>>> In terms of security configuration, you can follow the following guide:
>>> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/System_Admin_Guide.md#security-properties-in-bootstrapconf 
>>>
>>>
>>> About the different configuration files you mentioned:
>>> - bootstrap.conf is the main configuration file what we should update
>>> - minifi.properties is a generated properties file based on the
>>> bootstrap.conf
>>> - nifi.properties is the legacy configuration file of minifi
>>> - config.yaml is the legacy flow configuration, now we have the json 
>>> one
>>> what you have tried to import previously
>>>
>>> I hope it helps to set up your minfi agent.
>>>
>>> Regards,
>>> Peter
>>>
>>> Robert Cohen <ro...@anu.edu.au.invalid> ezt írta (időpont: 2025.
>>> jan. 7., K, 6:19):
>>>
>>>> I'm having a certain amount of trouble getting good info on
>>>> administration of minifi 2.
>>>>
>>>> The README file says thats documentation can be found at
>>>>
>>>> https://nifi.apache.org/minifi 
>>>> and
>>>> https://cwiki.apache.org/confluence/display/NIFI/MiNiFi 
>>>>
>>>>
>>>> neither of which URL works.
>>>>
>>>>
>>>> Im assuming the real documentation is at
>>>>
>>>>
>>>> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/System_Admin_Guide.md#config-file 
>>>>
>>>>
>>>>
>>>> But Im having some Im finding it a bit confusing.
>>>>
>>>> I want to configure the keystore/truststore and sensitive prop 
>>>> passwords.
>>>>
>>>> It appears I can configure them either in bootstrap.conf or
>>>> minifi.properties. Is either to be prefered over the other?
>>>>
>>>> The documention mentions nifi.properties and config.yml. But doesnt
>>>> mention minifi.properties.
>>>>
>>>> Im assuming the mentions of config.yml are leftover from minifi 1
>>>> documentation. Or does minifi 2 use it for backwards compatability.
>>>>
>>>> And I assume nifi.properties has been renamed to minifi.properties and
>>>> the documentation hasnt caught up.
>>>>
>>>>
>>>> Actually dumping strings out of the jar files, I can find instances of
>>>> nifi.properties. But I cant find any references to minifi.properties.
>>>>
>>>> However the tarball contains a minifi.properties file but not a
>>>> nifi.properties file.
>>>>
>>>> So Im really not sure what is going on.
>>>>
>>>>
>>>>
>>>>
>>>> On 6/1/2025 4:40 pm, Robert Cohen wrote:
>>>>> Ahh, it appears the functionality is there. Its just that the README
>>>>> file hasnt been updated,
>>>>>
>>>>> Theres now a transform-nifi command in addition to the transform-yml
>>>>> command.
>>>>>
>>>>>
>>>>> On 6/1/2025 2:59 pm, Robert Cohen wrote:
>>>>>> [You don't often get email from robert.cohen@anu.edu.au.invalid.
>>>>>> Learn why this is important at
>>>>>> https://aka.ms/LearnAboutSenderIdentification ]
>>>>>>
>>>>>> I had a look at the minifi toolkit.
>>>>>>
>>>>>> However, it only appears to contain functionality for converting 
>>>>>> legacy
>>>>>> minifi config to minifi 2 config
>>>>>>
>>>>>> Heres an excerpt from  the minifi-tookit 2.1.0 README file
>>>>>>
>>>>>>        Valid commands include:
>>>>>>        transform-yml: Transforms legacy MiNiFi flow config YAML into
>>>>>> MiNiFi flow config JSON
>>>>>>
>>>>>> ## Example
>>>>>> - You have an older version of MiNiFi located in
>>>>>> <legacy_minifi_directory>.
>>>>>> - You would like upgrade to the latest version of MiNiFi. You 
>>>>>> downloaded
>>>>>> and extracted the latest MiNiFi into <latest_minifi_directory>.
>>>>>> - Run the following command to migrate the flow and the bootstrap 
>>>>>> config
>>>>>> ```
>>>>>> ./config.sh transform-yml <legacy_minifi_directory>/conf/config.yml
>>>>>> <legacy_minifi_directory>/conf/bootstrap.conf
>>>>>> <latest_minifi_directory>/conf/flow.json.raw
>>>>>> <latest_minifi_directory>/conf/bootstrap.conf
>>>>>> ```
>>>>>>
>>>>>>
>>>>>> Is there functionality somewhere to convert a nifi 2.x flow file 
>>>>>> to a
>>>>>> minifi 2 flow file,
>>>>>>
>>>>>>
>>>>>> On 12/12/2024 6:48 pm, Kedvessy Péter wrote:
>>>>>>> [You don't often get email from pkedvessy@gmail.com. Learn why this
>>>>>>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>>>>>>>
>>>>>>> Hi Robert,
>>>>>>>
>>>>>>> Seems like the documentation has not been updated. I'm going to 
>>>>>>> file
>>>>>>> a JIRA
>>>>>>> ticket to apply doc update about how we can import NiFi 2 
>>>>>>> exported flow
>>>>>>> files into MiNiFi.
>>>>>>> In a nutshell: before moving the json file into minifi, we should
>>>>>>> convert
>>>>>>> it with minifi toolkit. Related ticket for more details:
>>>>>>> https://issues.apache.org/jira/browse/NIFI-12644 
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>> Peter
>>>>>>>
>>>>>>> Robert Cohen <ro...@anu.edu.au.invalid> ezt írta (időpont:
>>>> 2024.
>>>>>>> dec. 12., Cs, 2:11):
>>>>>>>
>>>>>>>> Im having trouble getting code into minifi 2 from nifi 2.
>>>>>>>>
>>>>>>>> The documentation here
>>>>>>>>
>>>>>>>>
>>>> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md 
>>>>
>>>>>>>>
>>>>>>>> says
>>>>>>>>
>>>>>>>> # Export the dataflow in JSON format.
>>>>>>>> # Move your new .json file to |minifi/conf|.
>>>>>>>> # Rename your .json file /flow.json.raw/.
>>>>>>>>
>>>>>>>>
>>>>>>>> Im using the option in nifi to "download flow definition"
>>>>>>>>
>>>>>>>> However when I load that into minifi, I get an error message.
>>>>>>>>
>>>>>>>> java.lang.Exception: Unable to load flow due to:
>>>>>>>> org.apache.nifi.controller.serialization.FlowSerializationException: 
>>>>>>>>
>>>>>>>> Could not parse flow as a VersionedDataflow
>>>>>>>>
>>>>>>>>
>>>>>>>> When I compare the file to the dummy flow file that ships with 
>>>>>>>> minifi.
>>>>>>>>
>>>>>>>> The file that ships with minifi starts with
>>>>>>>>
>>>>>>>> {
>>>>>>>>      "encodingVersion": {
>>>>>>>>        "majorVersion": 2,
>>>>>>>>        "minorVersion": 0
>>>>>>>>      },
>>>>>>>>
>>>>>>>>
>>>>>>>> But the export from nifi has
>>>>>>>>
>>>>>>>> {"flowContents":{"identifier":"a2ac574c-635d-3df5-baf0-6cbf4dd5bdec", 
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> So the 2 files dont appear to be the same format.
>>>>>>>>
>>>>>>>> Is there some other method of exporting code from nifi that Im
>>>>>>>> missing?
>>>>>>>>
>>>>>>>>

Re: importing code into minifi fron nifi

Posted by Robert Cohen <ro...@anu.edu.au.INVALID>.
I googled the error message.

I found a thread here which seems the same

https://community.cloudera.com/t5/Support-Questions/Unable-to-start-MiNiFi-because-of-an-error-caused-due-to-the/m-p/393219


One thing it suggests is editing the flow file and changing the 
following setting from 0.

|"maxTimerDrivenThreadCount":5 But when I do that minifi fails to start 
as it says the flow file fails validation Does the flow file contain a 
checksum? |


On 8/1/2025 1:27 pm, Robert Cohen wrote:
> Thanks for that. It clears some things up.
>
> Im not sure if the doco is supposed to be suitable for nifi 1/2 or 
> just nifi 2.
>
> But you might want to clarify in the doco whether config files are 
> just used by nifi 1. Just used by nifi 2. Or supported in nifi 2 for 
> legacy. But not recommended.
>
>
> Anyway, Ive put my code from a nifi 2.1.0 into minifi 2.1.0.
>
> I get the following error
>
> 2025-01-08 13:16:09,065 ERROR [MiNiFi logging handler] 
> org.apache.nifi.minifi.StdErr Failed to start flow service: Unable to 
> load flow due to: 
> org.apache.nifi.controller.serialization.FlowSynchronizationException: 
> java.lang.IllegalArgumentException: Cannot set max number of threads 
> to less than 1
>
> I attach the converted flow file
>
> I can't see any obvious parameter related to max number of threads.
>
>
>
> On 7/1/2025 6:56 pm, Kedvessy Péter wrote:
>> [You don't often get email from pkedvessy@gmail.com. Learn why this 
>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>>
>> Hi Robert,
>>
>> Thanks for reporting the documentation issues. I filed a JIRA to fix 
>> them:
>> https://issues.apache.org/jira/browse/NIFI-14093 
>>
>>
>> In terms of security configuration, you can follow the following guide:
>> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/System_Admin_Guide.md#security-properties-in-bootstrapconf 
>>
>>
>> About the different configuration files you mentioned:
>> - bootstrap.conf is the main configuration file what we should update
>> - minifi.properties is a generated properties file based on the
>> bootstrap.conf
>> - nifi.properties is the legacy configuration file of minifi
>> - config.yaml is the legacy flow configuration, now we have the json one
>> what you have tried to import previously
>>
>> I hope it helps to set up your minfi agent.
>>
>> Regards,
>> Peter
>>
>> Robert Cohen <ro...@anu.edu.au.invalid> ezt írta (időpont: 2025.
>> jan. 7., K, 6:19):
>>
>>> I'm having a certain amount of trouble getting good info on
>>> administration of minifi 2.
>>>
>>> The README file says thats documentation can be found at
>>>
>>> https://nifi.apache.org/minifi 
>>> and
>>> https://cwiki.apache.org/confluence/display/NIFI/MiNiFi 
>>>
>>>
>>> neither of which URL works.
>>>
>>>
>>> Im assuming the real documentation is at
>>>
>>>
>>> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/System_Admin_Guide.md#config-file 
>>>
>>>
>>>
>>> But Im having some Im finding it a bit confusing.
>>>
>>> I want to configure the keystore/truststore and sensitive prop 
>>> passwords.
>>>
>>> It appears I can configure them either in bootstrap.conf or
>>> minifi.properties. Is either to be prefered over the other?
>>>
>>> The documention mentions nifi.properties and config.yml. But doesnt
>>> mention minifi.properties.
>>>
>>> Im assuming the mentions of config.yml are leftover from minifi 1
>>> documentation. Or does minifi 2 use it for backwards compatability.
>>>
>>> And I assume nifi.properties has been renamed to minifi.properties and
>>> the documentation hasnt caught up.
>>>
>>>
>>> Actually dumping strings out of the jar files, I can find instances of
>>> nifi.properties. But I cant find any references to minifi.properties.
>>>
>>> However the tarball contains a minifi.properties file but not a
>>> nifi.properties file.
>>>
>>> So Im really not sure what is going on.
>>>
>>>
>>>
>>>
>>> On 6/1/2025 4:40 pm, Robert Cohen wrote:
>>>> Ahh, it appears the functionality is there. Its just that the README
>>>> file hasnt been updated,
>>>>
>>>> Theres now a transform-nifi command in addition to the transform-yml
>>>> command.
>>>>
>>>>
>>>> On 6/1/2025 2:59 pm, Robert Cohen wrote:
>>>>> [You don't often get email from robert.cohen@anu.edu.au.invalid.
>>>>> Learn why this is important at
>>>>> https://aka.ms/LearnAboutSenderIdentification ]
>>>>>
>>>>> I had a look at the minifi toolkit.
>>>>>
>>>>> However, it only appears to contain functionality for converting 
>>>>> legacy
>>>>> minifi config to minifi 2 config
>>>>>
>>>>> Heres an excerpt from  the minifi-tookit 2.1.0 README file
>>>>>
>>>>>        Valid commands include:
>>>>>        transform-yml: Transforms legacy MiNiFi flow config YAML into
>>>>> MiNiFi flow config JSON
>>>>>
>>>>> ## Example
>>>>> - You have an older version of MiNiFi located in
>>>>> <legacy_minifi_directory>.
>>>>> - You would like upgrade to the latest version of MiNiFi. You 
>>>>> downloaded
>>>>> and extracted the latest MiNiFi into <latest_minifi_directory>.
>>>>> - Run the following command to migrate the flow and the bootstrap 
>>>>> config
>>>>> ```
>>>>> ./config.sh transform-yml <legacy_minifi_directory>/conf/config.yml
>>>>> <legacy_minifi_directory>/conf/bootstrap.conf
>>>>> <latest_minifi_directory>/conf/flow.json.raw
>>>>> <latest_minifi_directory>/conf/bootstrap.conf
>>>>> ```
>>>>>
>>>>>
>>>>> Is there functionality somewhere to convert a nifi 2.x flow file to a
>>>>> minifi 2 flow file,
>>>>>
>>>>>
>>>>> On 12/12/2024 6:48 pm, Kedvessy Péter wrote:
>>>>>> [You don't often get email from pkedvessy@gmail.com. Learn why this
>>>>>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>>>>>>
>>>>>> Hi Robert,
>>>>>>
>>>>>> Seems like the documentation has not been updated. I'm going to file
>>>>>> a JIRA
>>>>>> ticket to apply doc update about how we can import NiFi 2 
>>>>>> exported flow
>>>>>> files into MiNiFi.
>>>>>> In a nutshell: before moving the json file into minifi, we should
>>>>>> convert
>>>>>> it with minifi toolkit. Related ticket for more details:
>>>>>> https://issues.apache.org/jira/browse/NIFI-12644 
>>>>>>
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Peter
>>>>>>
>>>>>> Robert Cohen <ro...@anu.edu.au.invalid> ezt írta (időpont:
>>> 2024.
>>>>>> dec. 12., Cs, 2:11):
>>>>>>
>>>>>>> Im having trouble getting code into minifi 2 from nifi 2.
>>>>>>>
>>>>>>> The documentation here
>>>>>>>
>>>>>>>
>>> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md 
>>>
>>>>>>>
>>>>>>> says
>>>>>>>
>>>>>>> # Export the dataflow in JSON format.
>>>>>>> # Move your new .json file to |minifi/conf|.
>>>>>>> # Rename your .json file /flow.json.raw/.
>>>>>>>
>>>>>>>
>>>>>>> Im using the option in nifi to "download flow definition"
>>>>>>>
>>>>>>> However when I load that into minifi, I get an error message.
>>>>>>>
>>>>>>> java.lang.Exception: Unable to load flow due to:
>>>>>>> org.apache.nifi.controller.serialization.FlowSerializationException: 
>>>>>>>
>>>>>>> Could not parse flow as a VersionedDataflow
>>>>>>>
>>>>>>>
>>>>>>> When I compare the file to the dummy flow file that ships with 
>>>>>>> minifi.
>>>>>>>
>>>>>>> The file that ships with minifi starts with
>>>>>>>
>>>>>>> {
>>>>>>>      "encodingVersion": {
>>>>>>>        "majorVersion": 2,
>>>>>>>        "minorVersion": 0
>>>>>>>      },
>>>>>>>
>>>>>>>
>>>>>>> But the export from nifi has
>>>>>>>
>>>>>>> {"flowContents":{"identifier":"a2ac574c-635d-3df5-baf0-6cbf4dd5bdec", 
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> So the 2 files dont appear to be the same format.
>>>>>>>
>>>>>>> Is there some other method of exporting code from nifi that Im
>>>>>>> missing?
>>>>>>>
>>>>>>>

Re: importing code into minifi fron nifi

Posted by Robert Cohen <ro...@anu.edu.au.INVALID>.
Thanks for that. It clears some things up.

Im not sure if the doco is supposed to be suitable for nifi 1/2 or just 
nifi 2.

But you might want to clarify in the doco whether config files are just 
used by nifi 1. Just used by nifi 2. Or supported in nifi 2 for legacy. 
But not recommended.


Anyway, Ive put my code from a nifi 2.1.0 into minifi 2.1.0.

I get the following error

2025-01-08 13:16:09,065 ERROR [MiNiFi logging handler] 
org.apache.nifi.minifi.StdErr Failed to start flow service: Unable to 
load flow due to: 
org.apache.nifi.controller.serialization.FlowSynchronizationException: 
java.lang.IllegalArgumentException: Cannot set max number of threads to 
less than 1

I attach the converted flow file

I can't see any obvious parameter related to max number of threads.



On 7/1/2025 6:56 pm, Kedvessy Péter wrote:
> [You don't often get email from pkedvessy@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
>
> Hi Robert,
>
> Thanks for reporting the documentation issues. I filed a JIRA to fix them:
> https://issues.apache.org/jira/browse/NIFI-14093
>
> In terms of security configuration, you can follow the following guide:
> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/System_Admin_Guide.md#security-properties-in-bootstrapconf
>
> About the different configuration files you mentioned:
> - bootstrap.conf is the main configuration file what we should update
> - minifi.properties is a generated properties file based on the
> bootstrap.conf
> - nifi.properties is the legacy configuration file of minifi
> - config.yaml is the legacy flow configuration, now we have the json one
> what you have tried to import previously
>
> I hope it helps to set up your minfi agent.
>
> Regards,
> Peter
>
> Robert Cohen <ro...@anu.edu.au.invalid> ezt írta (időpont: 2025.
> jan. 7., K, 6:19):
>
>> I'm having a certain amount of trouble getting good info on
>> administration of minifi 2.
>>
>> The README file says thats documentation can be found at
>>
>> https://nifi.apache.org/minifi and
>> https://cwiki.apache.org/confluence/display/NIFI/MiNiFi
>>
>> neither of which URL works.
>>
>>
>> Im assuming the real documentation is at
>>
>>
>> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/System_Admin_Guide.md#config-file
>>
>>
>> But Im having some Im finding it a bit confusing.
>>
>> I want to configure the keystore/truststore and sensitive prop passwords.
>>
>> It appears I can configure them either in bootstrap.conf or
>> minifi.properties. Is either to be prefered over the other?
>>
>> The documention mentions nifi.properties and config.yml. But doesnt
>> mention minifi.properties.
>>
>> Im assuming the mentions of config.yml are leftover from minifi 1
>> documentation. Or does minifi 2 use it for backwards compatability.
>>
>> And I assume nifi.properties has been renamed to minifi.properties and
>> the documentation hasnt caught up.
>>
>>
>> Actually dumping strings out of the jar files, I can find instances of
>> nifi.properties. But I cant find any references to minifi.properties.
>>
>> However the tarball contains a minifi.properties file but not a
>> nifi.properties file.
>>
>> So Im really not sure what is going on.
>>
>>
>>
>>
>> On 6/1/2025 4:40 pm, Robert Cohen wrote:
>>> Ahh, it appears the functionality is there. Its just that the README
>>> file hasnt been updated,
>>>
>>> Theres now a transform-nifi command in addition to the transform-yml
>>> command.
>>>
>>>
>>> On 6/1/2025 2:59 pm, Robert Cohen wrote:
>>>> [You don't often get email from robert.cohen@anu.edu.au.invalid.
>>>> Learn why this is important at
>>>> https://aka.ms/LearnAboutSenderIdentification ]
>>>>
>>>> I had a look at the minifi toolkit.
>>>>
>>>> However, it only appears to contain functionality for converting legacy
>>>> minifi config to minifi 2 config
>>>>
>>>> Heres an excerpt from  the minifi-tookit 2.1.0 README file
>>>>
>>>>        Valid commands include:
>>>>        transform-yml: Transforms legacy MiNiFi flow config YAML into
>>>> MiNiFi flow config JSON
>>>>
>>>> ## Example
>>>> - You have an older version of MiNiFi located in
>>>> <legacy_minifi_directory>.
>>>> - You would like upgrade to the latest version of MiNiFi. You downloaded
>>>> and extracted the latest MiNiFi into <latest_minifi_directory>.
>>>> - Run the following command to migrate the flow and the bootstrap config
>>>> ```
>>>> ./config.sh transform-yml <legacy_minifi_directory>/conf/config.yml
>>>> <legacy_minifi_directory>/conf/bootstrap.conf
>>>> <latest_minifi_directory>/conf/flow.json.raw
>>>> <latest_minifi_directory>/conf/bootstrap.conf
>>>> ```
>>>>
>>>>
>>>> Is there functionality somewhere to convert a nifi 2.x flow file to a
>>>> minifi 2 flow file,
>>>>
>>>>
>>>> On 12/12/2024 6:48 pm, Kedvessy Péter wrote:
>>>>> [You don't often get email from pkedvessy@gmail.com. Learn why this
>>>>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>>>>>
>>>>> Hi Robert,
>>>>>
>>>>> Seems like the documentation has not been updated. I'm going to file
>>>>> a JIRA
>>>>> ticket to apply doc update about how we can import NiFi 2 exported flow
>>>>> files into MiNiFi.
>>>>> In a nutshell: before moving the json file into minifi, we should
>>>>> convert
>>>>> it with minifi toolkit. Related ticket for more details:
>>>>> https://issues.apache.org/jira/browse/NIFI-12644
>>>>>
>>>>>
>>>>> Regards,
>>>>> Peter
>>>>>
>>>>> Robert Cohen <ro...@anu.edu.au.invalid> ezt írta (időpont:
>> 2024.
>>>>> dec. 12., Cs, 2:11):
>>>>>
>>>>>> Im having trouble getting code into minifi 2 from nifi 2.
>>>>>>
>>>>>> The documentation here
>>>>>>
>>>>>>
>> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md
>>>>>>
>>>>>> says
>>>>>>
>>>>>> # Export the dataflow in JSON format.
>>>>>> # Move your new .json file to |minifi/conf|.
>>>>>> # Rename your .json file /flow.json.raw/.
>>>>>>
>>>>>>
>>>>>> Im using the option in nifi to "download flow definition"
>>>>>>
>>>>>> However when I load that into minifi, I get an error message.
>>>>>>
>>>>>> java.lang.Exception: Unable to load flow due to:
>>>>>> org.apache.nifi.controller.serialization.FlowSerializationException:
>>>>>> Could not parse flow as a VersionedDataflow
>>>>>>
>>>>>>
>>>>>> When I compare the file to the dummy flow file that ships with minifi.
>>>>>>
>>>>>> The file that ships with minifi starts with
>>>>>>
>>>>>> {
>>>>>>      "encodingVersion": {
>>>>>>        "majorVersion": 2,
>>>>>>        "minorVersion": 0
>>>>>>      },
>>>>>>
>>>>>>
>>>>>> But the export from nifi has
>>>>>>
>>>>>> {"flowContents":{"identifier":"a2ac574c-635d-3df5-baf0-6cbf4dd5bdec",
>>>>>>
>>>>>>
>>>>>> So the 2 files dont appear to be the same format.
>>>>>>
>>>>>> Is there some other method of exporting code from nifi that Im
>>>>>> missing?
>>>>>>
>>>>>>

Re: importing code into minifi fron nifi

Posted by Kedvessy Péter <pk...@gmail.com>.
Hi Robert,

Thanks for reporting the documentation issues. I filed a JIRA to fix them:
https://issues.apache.org/jira/browse/NIFI-14093

In terms of security configuration, you can follow the following guide:
https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/System_Admin_Guide.md#security-properties-in-bootstrapconf

About the different configuration files you mentioned:
- bootstrap.conf is the main configuration file what we should update
- minifi.properties is a generated properties file based on the
bootstrap.conf
- nifi.properties is the legacy configuration file of minifi
- config.yaml is the legacy flow configuration, now we have the json one
what you have tried to import previously

I hope it helps to set up your minfi agent.

Regards,
Peter

Robert Cohen <ro...@anu.edu.au.invalid> ezt írta (időpont: 2025.
jan. 7., K, 6:19):

> I'm having a certain amount of trouble getting good info on
> administration of minifi 2.
>
> The README file says thats documentation can be found at
>
> https://nifi.apache.org/minifi and
> https://cwiki.apache.org/confluence/display/NIFI/MiNiFi
>
> neither of which URL works.
>
>
> Im assuming the real documentation is at
>
>
> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/System_Admin_Guide.md#config-file
>
>
> But Im having some Im finding it a bit confusing.
>
> I want to configure the keystore/truststore and sensitive prop passwords.
>
> It appears I can configure them either in bootstrap.conf or
> minifi.properties. Is either to be prefered over the other?
>
> The documention mentions nifi.properties and config.yml. But doesnt
> mention minifi.properties.
>
> Im assuming the mentions of config.yml are leftover from minifi 1
> documentation. Or does minifi 2 use it for backwards compatability.
>
> And I assume nifi.properties has been renamed to minifi.properties and
> the documentation hasnt caught up.
>
>
> Actually dumping strings out of the jar files, I can find instances of
> nifi.properties. But I cant find any references to minifi.properties.
>
> However the tarball contains a minifi.properties file but not a
> nifi.properties file.
>
> So Im really not sure what is going on.
>
>
>
>
> On 6/1/2025 4:40 pm, Robert Cohen wrote:
> > Ahh, it appears the functionality is there. Its just that the README
> > file hasnt been updated,
> >
> > Theres now a transform-nifi command in addition to the transform-yml
> > command.
> >
> >
> > On 6/1/2025 2:59 pm, Robert Cohen wrote:
> >> [You don't often get email from robert.cohen@anu.edu.au.invalid.
> >> Learn why this is important at
> >> https://aka.ms/LearnAboutSenderIdentification ]
> >>
> >> I had a look at the minifi toolkit.
> >>
> >> However, it only appears to contain functionality for converting legacy
> >> minifi config to minifi 2 config
> >>
> >> Heres an excerpt from  the minifi-tookit 2.1.0 README file
> >>
> >>       Valid commands include:
> >>       transform-yml: Transforms legacy MiNiFi flow config YAML into
> >> MiNiFi flow config JSON
> >>
> >> ## Example
> >> - You have an older version of MiNiFi located in
> >> <legacy_minifi_directory>.
> >> - You would like upgrade to the latest version of MiNiFi. You downloaded
> >> and extracted the latest MiNiFi into <latest_minifi_directory>.
> >> - Run the following command to migrate the flow and the bootstrap config
> >> ```
> >> ./config.sh transform-yml <legacy_minifi_directory>/conf/config.yml
> >> <legacy_minifi_directory>/conf/bootstrap.conf
> >> <latest_minifi_directory>/conf/flow.json.raw
> >> <latest_minifi_directory>/conf/bootstrap.conf
> >> ```
> >>
> >>
> >> Is there functionality somewhere to convert a nifi 2.x flow file to a
> >> minifi 2 flow file,
> >>
> >>
> >> On 12/12/2024 6:48 pm, Kedvessy Péter wrote:
> >>> [You don't often get email from pkedvessy@gmail.com. Learn why this
> >>> is important at https://aka.ms/LearnAboutSenderIdentification ]
> >>>
> >>> Hi Robert,
> >>>
> >>> Seems like the documentation has not been updated. I'm going to file
> >>> a JIRA
> >>> ticket to apply doc update about how we can import NiFi 2 exported flow
> >>> files into MiNiFi.
> >>> In a nutshell: before moving the json file into minifi, we should
> >>> convert
> >>> it with minifi toolkit. Related ticket for more details:
> >>> https://issues.apache.org/jira/browse/NIFI-12644
> >>>
> >>>
> >>> Regards,
> >>> Peter
> >>>
> >>> Robert Cohen <ro...@anu.edu.au.invalid> ezt írta (időpont:
> 2024.
> >>> dec. 12., Cs, 2:11):
> >>>
> >>>> Im having trouble getting code into minifi 2 from nifi 2.
> >>>>
> >>>> The documentation here
> >>>>
> >>>>
> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md
> >>>>
> >>>>
> >>>> says
> >>>>
> >>>> # Export the dataflow in JSON format.
> >>>> # Move your new .json file to |minifi/conf|.
> >>>> # Rename your .json file /flow.json.raw/.
> >>>>
> >>>>
> >>>> Im using the option in nifi to "download flow definition"
> >>>>
> >>>> However when I load that into minifi, I get an error message.
> >>>>
> >>>> java.lang.Exception: Unable to load flow due to:
> >>>> org.apache.nifi.controller.serialization.FlowSerializationException:
> >>>> Could not parse flow as a VersionedDataflow
> >>>>
> >>>>
> >>>> When I compare the file to the dummy flow file that ships with minifi.
> >>>>
> >>>> The file that ships with minifi starts with
> >>>>
> >>>> {
> >>>>     "encodingVersion": {
> >>>>       "majorVersion": 2,
> >>>>       "minorVersion": 0
> >>>>     },
> >>>>
> >>>>
> >>>> But the export from nifi has
> >>>>
> >>>> {"flowContents":{"identifier":"a2ac574c-635d-3df5-baf0-6cbf4dd5bdec",
> >>>>
> >>>>
> >>>> So the 2 files dont appear to be the same format.
> >>>>
> >>>> Is there some other method of exporting code from nifi that Im
> >>>> missing?
> >>>>
> >>>>
>

Re: importing code into minifi fron nifi

Posted by Robert Cohen <ro...@anu.edu.au.INVALID>.
I'm having a certain amount of trouble getting good info on 
administration of minifi 2.

The README file says thats documentation can be found at

https://nifi.apache.org/minifi and 
https://cwiki.apache.org/confluence/display/NIFI/MiNiFi

neither of which URL works.


Im assuming the real documentation is at

https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/System_Admin_Guide.md#config-file


But Im having some Im finding it a bit confusing.

I want to configure the keystore/truststore and sensitive prop passwords.

It appears I can configure them either in bootstrap.conf or 
minifi.properties. Is either to be prefered over the other?

The documention mentions nifi.properties and config.yml. But doesnt 
mention minifi.properties.

Im assuming the mentions of config.yml are leftover from minifi 1 
documentation. Or does minifi 2 use it for backwards compatability.

And I assume nifi.properties has been renamed to minifi.properties and 
the documentation hasnt caught up.


Actually dumping strings out of the jar files, I can find instances of 
nifi.properties. But I cant find any references to minifi.properties.

However the tarball contains a minifi.properties file but not a 
nifi.properties file.

So Im really not sure what is going on.




On 6/1/2025 4:40 pm, Robert Cohen wrote:
> Ahh, it appears the functionality is there. Its just that the README 
> file hasnt been updated,
>
> Theres now a transform-nifi command in addition to the transform-yml 
> command.
>
>
> On 6/1/2025 2:59 pm, Robert Cohen wrote:
>> [You don't often get email from robert.cohen@anu.edu.au.invalid. 
>> Learn why this is important at 
>> https://aka.ms/LearnAboutSenderIdentification ]
>>
>> I had a look at the minifi toolkit.
>>
>> However, it only appears to contain functionality for converting legacy
>> minifi config to minifi 2 config
>>
>> Heres an excerpt from  the minifi-tookit 2.1.0 README file
>>
>>       Valid commands include:
>>       transform-yml: Transforms legacy MiNiFi flow config YAML into
>> MiNiFi flow config JSON
>>
>> ## Example
>> - You have an older version of MiNiFi located in 
>> <legacy_minifi_directory>.
>> - You would like upgrade to the latest version of MiNiFi. You downloaded
>> and extracted the latest MiNiFi into <latest_minifi_directory>.
>> - Run the following command to migrate the flow and the bootstrap config
>> ```
>> ./config.sh transform-yml <legacy_minifi_directory>/conf/config.yml
>> <legacy_minifi_directory>/conf/bootstrap.conf
>> <latest_minifi_directory>/conf/flow.json.raw
>> <latest_minifi_directory>/conf/bootstrap.conf
>> ```
>>
>>
>> Is there functionality somewhere to convert a nifi 2.x flow file to a
>> minifi 2 flow file,
>>
>>
>> On 12/12/2024 6:48 pm, Kedvessy Péter wrote:
>>> [You don't often get email from pkedvessy@gmail.com. Learn why this 
>>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>>>
>>> Hi Robert,
>>>
>>> Seems like the documentation has not been updated. I'm going to file 
>>> a JIRA
>>> ticket to apply doc update about how we can import NiFi 2 exported flow
>>> files into MiNiFi.
>>> In a nutshell: before moving the json file into minifi, we should 
>>> convert
>>> it with minifi toolkit. Related ticket for more details:
>>> https://issues.apache.org/jira/browse/NIFI-12644 
>>>
>>>
>>> Regards,
>>> Peter
>>>
>>> Robert Cohen <ro...@anu.edu.au.invalid> ezt írta (időpont: 2024.
>>> dec. 12., Cs, 2:11):
>>>
>>>> Im having trouble getting code into minifi 2 from nifi 2.
>>>>
>>>> The documentation here
>>>>
>>>> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md 
>>>>
>>>>
>>>> says
>>>>
>>>> # Export the dataflow in JSON format.
>>>> # Move your new .json file to |minifi/conf|.
>>>> # Rename your .json file /flow.json.raw/.
>>>>
>>>>
>>>> Im using the option in nifi to "download flow definition"
>>>>
>>>> However when I load that into minifi, I get an error message.
>>>>
>>>> java.lang.Exception: Unable to load flow due to:
>>>> org.apache.nifi.controller.serialization.FlowSerializationException:
>>>> Could not parse flow as a VersionedDataflow
>>>>
>>>>
>>>> When I compare the file to the dummy flow file that ships with minifi.
>>>>
>>>> The file that ships with minifi starts with
>>>>
>>>> {
>>>>     "encodingVersion": {
>>>>       "majorVersion": 2,
>>>>       "minorVersion": 0
>>>>     },
>>>>
>>>>
>>>> But the export from nifi has
>>>>
>>>> {"flowContents":{"identifier":"a2ac574c-635d-3df5-baf0-6cbf4dd5bdec",
>>>>
>>>>
>>>> So the 2 files dont appear to be the same format.
>>>>
>>>> Is there some other method of exporting code from nifi that Im 
>>>> missing?
>>>>
>>>>

Re: importing code into minifi fron nifi

Posted by Robert Cohen <ro...@anu.edu.au.INVALID>.
Ahh, it appears the functionality is there. Its just that the README 
file hasnt been updated,

Theres now a transform-nifi command in addition to the transform-yml 
command.


On 6/1/2025 2:59 pm, Robert Cohen wrote:
> [You don't often get email from robert.cohen@anu.edu.au.invalid. Learn 
> why this is important at https://aka.ms/LearnAboutSenderIdentification ]
>
> I had a look at the minifi toolkit.
>
> However, it only appears to contain functionality for converting legacy
> minifi config to minifi 2 config
>
> Heres an excerpt from  the minifi-tookit 2.1.0 README file
>
>       Valid commands include:
>       transform-yml: Transforms legacy MiNiFi flow config YAML into
> MiNiFi flow config JSON
>
> ## Example
> - You have an older version of MiNiFi located in 
> <legacy_minifi_directory>.
> - You would like upgrade to the latest version of MiNiFi. You downloaded
> and extracted the latest MiNiFi into <latest_minifi_directory>.
> - Run the following command to migrate the flow and the bootstrap config
> ```
> ./config.sh transform-yml <legacy_minifi_directory>/conf/config.yml
> <legacy_minifi_directory>/conf/bootstrap.conf
> <latest_minifi_directory>/conf/flow.json.raw
> <latest_minifi_directory>/conf/bootstrap.conf
> ```
>
>
> Is there functionality somewhere to convert a nifi 2.x flow file to a
> minifi 2 flow file,
>
>
> On 12/12/2024 6:48 pm, Kedvessy Péter wrote:
>> [You don't often get email from pkedvessy@gmail.com. Learn why this 
>> is important at https://aka.ms/LearnAboutSenderIdentification ]
>>
>> Hi Robert,
>>
>> Seems like the documentation has not been updated. I'm going to file 
>> a JIRA
>> ticket to apply doc update about how we can import NiFi 2 exported flow
>> files into MiNiFi.
>> In a nutshell: before moving the json file into minifi, we should 
>> convert
>> it with minifi toolkit. Related ticket for more details:
>> https://issues.apache.org/jira/browse/NIFI-12644 
>>
>>
>> Regards,
>> Peter
>>
>> Robert Cohen <ro...@anu.edu.au.invalid> ezt írta (időpont: 2024.
>> dec. 12., Cs, 2:11):
>>
>>> Im having trouble getting code into minifi 2 from nifi 2.
>>>
>>> The documentation here
>>>
>>> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md 
>>>
>>>
>>> says
>>>
>>> # Export the dataflow in JSON format.
>>> # Move your new .json file to |minifi/conf|.
>>> # Rename your .json file /flow.json.raw/.
>>>
>>>
>>> Im using the option in nifi to "download flow definition"
>>>
>>> However when I load that into minifi, I get an error message.
>>>
>>> java.lang.Exception: Unable to load flow due to:
>>> org.apache.nifi.controller.serialization.FlowSerializationException:
>>> Could not parse flow as a VersionedDataflow
>>>
>>>
>>> When I compare the file to the dummy flow file that ships with minifi.
>>>
>>> The file that ships with minifi starts with
>>>
>>> {
>>>     "encodingVersion": {
>>>       "majorVersion": 2,
>>>       "minorVersion": 0
>>>     },
>>>
>>>
>>> But the export from nifi has
>>>
>>> {"flowContents":{"identifier":"a2ac574c-635d-3df5-baf0-6cbf4dd5bdec",
>>>
>>>
>>> So the 2 files dont appear to be the same format.
>>>
>>> Is there some other method of exporting code from nifi that Im missing?
>>>
>>>

Re: importing code into minifi fron nifi

Posted by Robert Cohen <ro...@anu.edu.au.INVALID>.
I had a look at the minifi toolkit.

However, it only appears to contain functionality for converting legacy 
minifi config to minifi 2 config

Heres an excerpt from  the minifi-tookit 2.1.0 README file

       Valid commands include:
       transform-yml: Transforms legacy MiNiFi flow config YAML into 
MiNiFi flow config JSON

## Example
- You have an older version of MiNiFi located in <legacy_minifi_directory>.
- You would like upgrade to the latest version of MiNiFi. You downloaded 
and extracted the latest MiNiFi into <latest_minifi_directory>.
- Run the following command to migrate the flow and the bootstrap config
```
./config.sh transform-yml <legacy_minifi_directory>/conf/config.yml 
<legacy_minifi_directory>/conf/bootstrap.conf 
<latest_minifi_directory>/conf/flow.json.raw 
<latest_minifi_directory>/conf/bootstrap.conf
```


Is there functionality somewhere to convert a nifi 2.x flow file to a 
minifi 2 flow file,


On 12/12/2024 6:48 pm, Kedvessy Péter wrote:
> [You don't often get email from pkedvessy@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
>
> Hi Robert,
>
> Seems like the documentation has not been updated. I'm going to file a JIRA
> ticket to apply doc update about how we can import NiFi 2 exported flow
> files into MiNiFi.
> In a nutshell: before moving the json file into minifi, we should convert
> it with minifi toolkit. Related ticket for more details:
> https://issues.apache.org/jira/browse/NIFI-12644
>
> Regards,
> Peter
>
> Robert Cohen <ro...@anu.edu.au.invalid> ezt írta (időpont: 2024.
> dec. 12., Cs, 2:11):
>
>> Im having trouble getting code into minifi 2 from nifi 2.
>>
>> The documentation here
>>
>> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md
>>
>> says
>>
>> # Export the dataflow in JSON format.
>> # Move your new .json file to |minifi/conf|.
>> # Rename your .json file /flow.json.raw/.
>>
>>
>> Im using the option in nifi to "download flow definition"
>>
>> However when I load that into minifi, I get an error message.
>>
>> java.lang.Exception: Unable to load flow due to:
>> org.apache.nifi.controller.serialization.FlowSerializationException:
>> Could not parse flow as a VersionedDataflow
>>
>>
>> When I compare the file to the dummy flow file that ships with minifi.
>>
>> The file that ships with minifi starts with
>>
>> {
>>     "encodingVersion": {
>>       "majorVersion": 2,
>>       "minorVersion": 0
>>     },
>>
>>
>> But the export from nifi has
>>
>> {"flowContents":{"identifier":"a2ac574c-635d-3df5-baf0-6cbf4dd5bdec",
>>
>>
>> So the 2 files dont appear to be the same format.
>>
>> Is there some other method of exporting code from nifi that Im missing?
>>
>>

Re: importing code into minifi fron nifi

Posted by Kedvessy Péter <pk...@gmail.com>.
Hi Robert,

Seems like the documentation has not been updated. I'm going to file a JIRA
ticket to apply doc update about how we can import NiFi 2 exported flow
files into MiNiFi.
In a nutshell: before moving the json file into minifi, we should convert
it with minifi toolkit. Related ticket for more details:
https://issues.apache.org/jira/browse/NIFI-12644

Regards,
Peter

Robert Cohen <ro...@anu.edu.au.invalid> ezt írta (időpont: 2024.
dec. 12., Cs, 2:11):

> Im having trouble getting code into minifi 2 from nifi 2.
>
> The documentation here
>
> https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md
>
> says
>
> # Export the dataflow in JSON format.
> # Move your new .json file to |minifi/conf|.
> # Rename your .json file /flow.json.raw/.
>
>
> Im using the option in nifi to "download flow definition"
>
> However when I load that into minifi, I get an error message.
>
> java.lang.Exception: Unable to load flow due to:
> org.apache.nifi.controller.serialization.FlowSerializationException:
> Could not parse flow as a VersionedDataflow
>
>
> When I compare the file to the dummy flow file that ships with minifi.
>
> The file that ships with minifi starts with
>
> {
>    "encodingVersion": {
>      "majorVersion": 2,
>      "minorVersion": 0
>    },
>
>
> But the export from nifi has
>
> {"flowContents":{"identifier":"a2ac574c-635d-3df5-baf0-6cbf4dd5bdec",
>
>
> So the 2 files dont appear to be the same format.
>
> Is there some other method of exporting code from nifi that Im missing?
>
>