You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by David Gallagher <dg...@CleverDevices.com> on 2024/09/28 17:20:55 UTC

Nifi 2.0.0-M4: Python Processor relationships

Hi - I'm playing around with creating a native Python processor in M4. I've been able to get them to show up as new Processor options, but they don't have any Output relationships. /python/extensions is empty in M4, but I moved one of the extensions from M2 into the directory and same problem. I don't see anything obvious in the Python development docs. Is this a known issue?

Thanks,

Dave

Re: Nifi 2.0.0-M4: Python Processor relationships

Posted by Joe Witt <jo...@gmail.com>.
awesome!  good luck

On Sat, Sep 28, 2024 at 4:27 PM David Gallagher <
dgallagher@cleverdevices.com> wrote:

> Success with 3.11! Thanks, Joe - appreciate the assist! I can see
> env-creation-complete.txt in the work directory.
> ------------------------------
> *From:* Joe Witt <jo...@gmail.com>
> *Sent:* Saturday, September 28, 2024 4:01 PM
> *To:* users@nifi.apache.org <us...@nifi.apache.org>
> *Subject:* Re: Nifi 2.0.0-M4: Python Processor relationships
>
> Are you using the docker image or a plain install?
>
> What version of Python do you have?  I believe it requires 3.11.something
> or newer.
>
> Thanks
>
> On Sat, Sep 28, 2024 at 12:53 PM David Gallagher <
> dgallagher@cleverdevices.com> wrote:
>
> I'm getting this error in the logs:
>
> 2024-09-28 15:50:30,496 INFO [Initialize TransformOpenskyStates]
> org.apache.nifi.py4j.PythonProcess Creating Python Virtual Environment
> ./work/python/extensions/TransformOpenskyStates/0.0.1-SNAPSHOT
>         at
> org.apache.nifi.python.processor.PythonProcessorProxy.lambda$new$0(PythonProcessorProxy.java:78)
> Caused by: java.io.IOException: Failed to create Python Environment
> ./work/python/extensions/TransformOpenskyStates/0.0.1-SNAPSHOT: process
> existed with code 1
>
> I can what appears to be a full environment in the work folder, though, so
> I don't know what's wrong.
> ------------------------------
> *From:* Joe Witt <jo...@gmail.com>
> *Sent:* Saturday, September 28, 2024 2:01 PM
> *To:* users@nifi.apache.org <us...@nifi.apache.org>
> *Subject:* Re: Nifi 2.0.0-M4: Python Processor relationships
>
> Dave
>
> Check out this tutorial.  I just followed it as well and got that
> component working and then built another example off of it.
>
> https://devcenter.datavolo.io/tutorials/build-python-transform/#0
>
> Thanks
> Joe
>
> On Sat, Sep 28, 2024 at 10:22 AM David Gallagher <
> dgallagher@cleverdevices.com> wrote:
>
> Hi - I'm playing around with creating a native Python processor in M4.
> I've been able to get them to show up as new Processor options, but they
> don't have any Output relationships. /python/extensions is empty in M4, but
> I moved one of the extensions from M2 into the directory and same problem.
> I don't see anything obvious in the Python development docs. Is this a
> known issue?
>
> Thanks,
>
> Dave
>
>

Re: Nifi 2.0.0-M4: Python Processor relationships

Posted by David Gallagher <dg...@CleverDevices.com>.
Success with 3.11! Thanks, Joe - appreciate the assist! I can see env-creation-complete.txt in the work directory.
________________________________
From: Joe Witt <jo...@gmail.com>
Sent: Saturday, September 28, 2024 4:01 PM
To: users@nifi.apache.org <us...@nifi.apache.org>
Subject: Re: Nifi 2.0.0-M4: Python Processor relationships

Are you using the docker image or a plain install? What version of Python do you have? I believe it requires 3.11.something or newer. Thanks  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌
External (joe.witt@gmail.com<ma...@gmail.com>)
  Report This Email<https://protection.inkyphishfence.com/report?id=c2tvdXQtY2xldmVyLWRldmljZXMvZGdhbGxhZ2hlckBjbGV2ZXJkZXZpY2VzLmNvbS9hN2U2ODYwZWU2NWFlMDIxNDdiYWUwNDY2OGVlZWZjOS8xNzI3NTUzNzM5LjI4#key=d0314affc5fee1d6f7a072cd59ac32b0>  FAQ<https://getskout.com/emailprotectionfaq/>  Skout Email Protection<https://getskout.com/emailprotection/>

Are you using the docker image or a plain install?

What version of Python do you have?  I believe it requires 3.11.something or newer.

Thanks

On Sat, Sep 28, 2024 at 12:53 PM David Gallagher <dg...@cleverdevices.com>> wrote:
I'm getting this error in the logs:

2024-09-28 15:50:30,496 INFO [Initialize TransformOpenskyStates] org.apache.nifi.py4j.PythonProcess Creating Python Virtual Environment ./work/python/extensions/TransformOpenskyStates/0.0.1-SNAPSHOT
        at org.apache.nifi.python.processor.PythonProcessorProxy.lambda$new$0(PythonProcessorProxy.java:78)
Caused by: java.io.IOException: Failed to create Python Environment ./work/python/extensions/TransformOpenskyStates/0.0.1-SNAPSHOT: process existed with code 1

I can what appears to be a full environment in the work folder, though, so I don't know what's wrong.
________________________________
From: Joe Witt <jo...@gmail.com>>
Sent: Saturday, September 28, 2024 2:01 PM
To: users@nifi.apache.org<ma...@nifi.apache.org> <us...@nifi.apache.org>>
Subject: Re: Nifi 2.0.0-M4: Python Processor relationships

Dave

Check out this tutorial.  I just followed it as well and got that component working and then built another example off of it.

https://devcenter.datavolo.io/tutorials/build-python-transform/#0<https://shared.outlook.inky.com/link?domain=devcenter.datavolo.io&t=h.eJwtzUFuwjAQheGrILNt7BCIHVhxlcEeEsMkg-xxqqri7sRqV2_xSf_7VSWRuuzUJPLKF2MCrh4XwaQDCKxMrCMbKcIpAmVzK5FC8_qRiZdGEiz5zmk2-1Z97dSzlvKTizSecMXUbLnoMZswAhGME6brn_yD9jwbcGgH2yLaHrDtDid32_Zk7YCId382B9e5vj-641l3Qz3CevRg1N9R5DrOEKmWKoWNlkL0_gC8a0iH.MEUCIDtcWdTT2PiJJTJmSP9Iy3oGeRuthJUV7bZc9cZLGEBuAiEAqsHFj5IzJFYqE8sXLZQawjbmHsLlxQpBG5MAgnD-_nM>

Thanks
Joe

On Sat, Sep 28, 2024 at 10:22 AM David Gallagher <dg...@cleverdevices.com>> wrote:
Hi - I'm playing around with creating a native Python processor in M4. I've been able to get them to show up as new Processor options, but they don't have any Output relationships. /python/extensions is empty in M4, but I moved one of the extensions from M2 into the directory and same problem. I don't see anything obvious in the Python development docs. Is this a known issue?

Thanks,

Dave

Re: Nifi 2.0.0-M4: Python Processor relationships

Posted by David Gallagher <dg...@CleverDevices.com>.
Plain installation on a Ubuntu VM. I've got 3.10, I can try with 3.11 and see if it makes a difference.
________________________________
From: Joe Witt <jo...@gmail.com>
Sent: Saturday, September 28, 2024 4:01 PM
To: users@nifi.apache.org <us...@nifi.apache.org>
Subject: Re: Nifi 2.0.0-M4: Python Processor relationships

Are you using the docker image or a plain install? What version of Python do you have? I believe it requires 3.11.something or newer. Thanks  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌  ‌
External (joe.witt@gmail.com<ma...@gmail.com>)
  Report This Email<https://protection.inkyphishfence.com/report?id=c2tvdXQtY2xldmVyLWRldmljZXMvZGdhbGxhZ2hlckBjbGV2ZXJkZXZpY2VzLmNvbS9hN2U2ODYwZWU2NWFlMDIxNDdiYWUwNDY2OGVlZWZjOS8xNzI3NTUzNzM5LjI4#key=d0314affc5fee1d6f7a072cd59ac32b0>  FAQ<https://getskout.com/emailprotectionfaq/>  Skout Email Protection<https://getskout.com/emailprotection/>

Are you using the docker image or a plain install?

What version of Python do you have?  I believe it requires 3.11.something or newer.

Thanks

On Sat, Sep 28, 2024 at 12:53 PM David Gallagher <dg...@cleverdevices.com>> wrote:
I'm getting this error in the logs:

2024-09-28 15:50:30,496 INFO [Initialize TransformOpenskyStates] org.apache.nifi.py4j.PythonProcess Creating Python Virtual Environment ./work/python/extensions/TransformOpenskyStates/0.0.1-SNAPSHOT
        at org.apache.nifi.python.processor.PythonProcessorProxy.lambda$new$0(PythonProcessorProxy.java:78)
Caused by: java.io.IOException: Failed to create Python Environment ./work/python/extensions/TransformOpenskyStates/0.0.1-SNAPSHOT: process existed with code 1

I can what appears to be a full environment in the work folder, though, so I don't know what's wrong.
________________________________
From: Joe Witt <jo...@gmail.com>>
Sent: Saturday, September 28, 2024 2:01 PM
To: users@nifi.apache.org<ma...@nifi.apache.org> <us...@nifi.apache.org>>
Subject: Re: Nifi 2.0.0-M4: Python Processor relationships

Dave

Check out this tutorial.  I just followed it as well and got that component working and then built another example off of it.

https://devcenter.datavolo.io/tutorials/build-python-transform/#0<https://shared.outlook.inky.com/link?domain=devcenter.datavolo.io&t=h.eJwtzUFuwjAQheGrILNt7BCIHVhxlcEeEsMkg-xxqqri7sRqV2_xSf_7VSWRuuzUJPLKF2MCrh4XwaQDCKxMrCMbKcIpAmVzK5FC8_qRiZdGEiz5zmk2-1Z97dSzlvKTizSecMXUbLnoMZswAhGME6brn_yD9jwbcGgH2yLaHrDtDid32_Zk7YCId382B9e5vj-641l3Qz3CevRg1N9R5DrOEKmWKoWNlkL0_gC8a0iH.MEUCIDtcWdTT2PiJJTJmSP9Iy3oGeRuthJUV7bZc9cZLGEBuAiEAqsHFj5IzJFYqE8sXLZQawjbmHsLlxQpBG5MAgnD-_nM>

Thanks
Joe

On Sat, Sep 28, 2024 at 10:22 AM David Gallagher <dg...@cleverdevices.com>> wrote:
Hi - I'm playing around with creating a native Python processor in M4. I've been able to get them to show up as new Processor options, but they don't have any Output relationships. /python/extensions is empty in M4, but I moved one of the extensions from M2 into the directory and same problem. I don't see anything obvious in the Python development docs. Is this a known issue?

Thanks,

Dave

Re: Nifi 2.0.0-M4: Python Processor relationships

Posted by Joe Witt <jo...@gmail.com>.
Are you using the docker image or a plain install?

What version of Python do you have?  I believe it requires 3.11.something
or newer.

Thanks

On Sat, Sep 28, 2024 at 12:53 PM David Gallagher <
dgallagher@cleverdevices.com> wrote:

> I'm getting this error in the logs:
>
> 2024-09-28 15:50:30,496 INFO [Initialize TransformOpenskyStates]
> org.apache.nifi.py4j.PythonProcess Creating Python Virtual Environment
> ./work/python/extensions/TransformOpenskyStates/0.0.1-SNAPSHOT
>         at
> org.apache.nifi.python.processor.PythonProcessorProxy.lambda$new$0(PythonProcessorProxy.java:78)
> Caused by: java.io.IOException: Failed to create Python Environment
> ./work/python/extensions/TransformOpenskyStates/0.0.1-SNAPSHOT: process
> existed with code 1
>
> I can what appears to be a full environment in the work folder, though, so
> I don't know what's wrong.
> ------------------------------
> *From:* Joe Witt <jo...@gmail.com>
> *Sent:* Saturday, September 28, 2024 2:01 PM
> *To:* users@nifi.apache.org <us...@nifi.apache.org>
> *Subject:* Re: Nifi 2.0.0-M4: Python Processor relationships
>
> Dave
>
> Check out this tutorial.  I just followed it as well and got that
> component working and then built another example off of it.
>
> https://devcenter.datavolo.io/tutorials/build-python-transform/#0
>
> Thanks
> Joe
>
> On Sat, Sep 28, 2024 at 10:22 AM David Gallagher <
> dgallagher@cleverdevices.com> wrote:
>
> Hi - I'm playing around with creating a native Python processor in M4.
> I've been able to get them to show up as new Processor options, but they
> don't have any Output relationships. /python/extensions is empty in M4, but
> I moved one of the extensions from M2 into the directory and same problem.
> I don't see anything obvious in the Python development docs. Is this a
> known issue?
>
> Thanks,
>
> Dave
>
>

Re: Nifi 2.0.0-M4: Python Processor relationships

Posted by David Gallagher <dg...@CleverDevices.com>.
I'm getting this error in the logs:

2024-09-28 15:50:30,496 INFO [Initialize TransformOpenskyStates] org.apache.nifi.py4j.PythonProcess Creating Python Virtual Environment ./work/python/extensions/TransformOpenskyStates/0.0.1-SNAPSHOT
        at org.apache.nifi.python.processor.PythonProcessorProxy.lambda$new$0(PythonProcessorProxy.java:78)
Caused by: java.io.IOException: Failed to create Python Environment ./work/python/extensions/TransformOpenskyStates/0.0.1-SNAPSHOT: process existed with code 1

I can what appears to be a full environment in the work folder, though, so I don't know what's wrong.
________________________________
From: Joe Witt <jo...@gmail.com>
Sent: Saturday, September 28, 2024 2:01 PM
To: users@nifi.apache.org <us...@nifi.apache.org>
Subject: Re: Nifi 2.0.0-M4: Python Processor relationships

Dave Check out this tutorial. I just followed it as well and got that component working and then built another example off of it. https://devcenter.datavolo.io/tutorials/build-python-transform/#0 Than
External (joe.witt@gmail.com<ma...@gmail.com>)
  Report This Email<https://protection.inkyphishfence.com/report?id=c2tvdXQtY2xldmVyLWRldmljZXMvZGdhbGxhZ2hlckBjbGV2ZXJkZXZpY2VzLmNvbS8zNGY5NTk0YmRmNjBjNDg3MWE2NjU4YWUxNzFmN2QyMi8xNzI3NTQ2NTI5LjYz#key=78ecd931166f1492bd3756c3f4025235>  FAQ<https://getskout.com/emailprotectionfaq/>  Skout Email Protection<https://getskout.com/emailprotection/>

Dave

Check out this tutorial.  I just followed it as well and got that component working and then built another example off of it.

https://devcenter.datavolo.io/tutorials/build-python-transform/#0<https://shared.outlook.inky.com/link?domain=devcenter.datavolo.io&t=h.eJwtzUFuwjAQQNGrIHfb2CQkNmHFVQZ7krhMMsgep0Kody9Wu37S_y9VEqnLQS0ij3wxJuDucRNMOoDAzsQ6spEinCJQNrcSKTSPpyy8NZJgyxOn1Xwc1edB3Wsp37lI4wl3TM07Fz1mE2YggnnBdP2Tf9CeV3Pqp3EY-1uY7NH3Z9eCtcMZsHXt5ELXmdZ1bujt0I3anuoI6-iLUX9Hkeu8QqRaqhTetBWin1-MykfJ.MEUCIHS975J_4AxsMvzH4KZR8uKcszsYTP-wLybGs0kPHEm6AiEA8Bm_MYV2a-3jV9ierZgeI2Y4h4O6HLZeJozl17I2xKo>

Thanks
Joe

On Sat, Sep 28, 2024 at 10:22 AM David Gallagher <dg...@cleverdevices.com>> wrote:
Hi - I'm playing around with creating a native Python processor in M4. I've been able to get them to show up as new Processor options, but they don't have any Output relationships. /python/extensions is empty in M4, but I moved one of the extensions from M2 into the directory and same problem. I don't see anything obvious in the Python development docs. Is this a known issue?

Thanks,

Dave

Re: Nifi 2.0.0-M4: Python Processor relationships

Posted by Joe Witt <jo...@gmail.com>.
Dave

Check out this tutorial.  I just followed it as well and got that component
working and then built another example off of it.

https://devcenter.datavolo.io/tutorials/build-python-transform/#0

Thanks
Joe

On Sat, Sep 28, 2024 at 10:22 AM David Gallagher <
dgallagher@cleverdevices.com> wrote:

> Hi - I'm playing around with creating a native Python processor in M4.
> I've been able to get them to show up as new Processor options, but they
> don't have any Output relationships. /python/extensions is empty in M4, but
> I moved one of the extensions from M2 into the directory and same problem.
> I don't see anything obvious in the Python development docs. Is this a
> known issue?
>
> Thanks,
>
> Dave
>