You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Rick Braddy <rb...@softnas.com> on 2015/08/30 19:34:15 UTC
Getting started developing/debugging Nifi processors
Hi,
The Nifi Developer's Guide is a great read with lots of useful information. However, as a relative noob to Nifi development, I still have a few questions that seemed unanswered (or perhaps I overlooked):
1. Recommended IDE for Nifi - given Nifi development involves both Java and Maven, is there a recommended or preferred IDE (e.g., NetBeans vs. Eclipse - other?) that's considered better? I realize this is also probably personal preference, but would like to know what the core Nifi team uses and recommends for debugging, both local and remote Nifi instances.
2. Debugging mode - given Nifi often runs as a Linux service, for debugging are you typically attaching to one or more running processes or launching Nifi separately to debug it? If attaching, what does one attach to, given many running java processes come and go?
3. Processor development vs. NAR packaging - when developing and debugging, given the run-time Nifi is installed in a different directory tree vs. the source tree, what is the most productive approach to inline development and debugging (vs. making changes, packaging and then installing NAR package into run-time area?
Is there any documentation or blog posts covering the above? (Google and I didn't find it)
Thanks in advance.
Rick
RE: Getting started developing/debugging Nifi processors
Posted by Rick Braddy <rb...@softnas.com>.
Thanks for the additional details, Matt. That worked!
I am able to set breakpoints from NetBeans within ExecuteProcess onTrigger() method, so I'm on my way!!
Appreciate the support.
Rick
-----Original Message-----
From: Matt Gilman [mailto:matt.c.gilman@gmail.com]
Sent: Sunday, August 30, 2015 1:31 PM
To: dev@nifi.apache.org
Subject: Re: Getting started developing/debugging Nifi processors
Rick,
You should be able to run NiFi from where you installed it from the .tar.gz. Once you've enabled remote debugging (via the line in the
bootstrap.conf) you'll be able to attach the remote debugger in your IDE.
The remote debugger will need to be configured with the port, though I believe 8000 is the default. The only other nugget here is the suspend flag. By default it's set to 'n', but you can change to 'y' if you need to debug something that is in your processors initialization logic. It will suspend launching until the remote debugger is attached.
Matt
On Sun, Aug 30, 2015 at 3:41 PM, Rick Braddy <rb...@softnas.com> wrote:
> Joe,
>
> Thanks for that information. I will start with NetBeans, as I've used
> it before on some other projects.
>
> On #2, during development and debugging are you running Nifi within
> the source tree or in a separate target tree where it's been installed
> from the .tar.gz?
>
> Rick
>
> -----Original Message-----
> From: Joe Witt [mailto:joe.witt@gmail.com]
> Sent: Sunday, August 30, 2015 10:43 AM
> To: dev@nifi.apache.org
> Subject: Re: Getting started developing/debugging Nifi processors
>
> Rick
>
> These are great and should be turned into a FAQ for developers so they
> can easily get information on these. Some aspects of these are
> covered in the developer guide but not clearly or completely.
>
> #1 I know folks on the team that use Eclipse, IntelliJ, and Netbeans
> and probably in that order of popularity. I personally prefer
> Netbeans because the integration with Maven just flat out works -
> flawlessly. That is
> *definitely* a personal preference thing though.
>
> #2 You can really easily bind to a running NiFi instance or cause
> binding to occur as soon as NiFi starts up. This is so you can truly
> debug a running instance including stepping through code. In the
> conf/boostrap.conf file check these two lines out:
>
> # Enable Remote Debugging
>
> #java.arg.debug=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,
> address=8000
>
> By uncommenting that second line you can attach a debugger at runtime
> to port 8000 and step through the code. Super cool.
>
> #3 By far the most efficient way to do this is through effective unit
> testing. The mock framework makes this work out quite well. Now, it
> is challenging at times to effectively unit test things which interact
> with external services. For those cases where you truly must update
> the version of nars in an actual instance of NiFi you may want to just
> script out commands to copy over the old nar and restart nifi.
>
> Thanks
> Joe
>
> On Sun, Aug 30, 2015 at 1:34 PM, Rick Braddy <rb...@softnas.com> wrote:
> > Hi,
> >
> > The Nifi Developer's Guide is a great read with lots of useful
> information. However, as a relative noob to Nifi development, I still
> have a few questions that seemed unanswered (or perhaps I overlooked):
> >
> >
> > 1. Recommended IDE for Nifi - given Nifi development involves both
> Java and Maven, is there a recommended or preferred IDE (e.g., NetBeans vs.
> Eclipse - other?) that's considered better? I realize this is also
> probably personal preference, but would like to know what the core
> Nifi team uses and recommends for debugging, both local and remote
> Nifi instances.
> >
> >
> > 2. Debugging mode - given Nifi often runs as a Linux service, for
> debugging are you typically attaching to one or more running processes
> or launching Nifi separately to debug it? If attaching, what does one
> attach to, given many running java processes come and go?
> >
> >
> > 3. Processor development vs. NAR packaging - when developing and
> debugging, given the run-time Nifi is installed in a different
> directory tree vs. the source tree, what is the most productive
> approach to inline development and debugging (vs. making changes,
> packaging and then installing NAR package into run-time area?
> >
> > Is there any documentation or blog posts covering the above?
> > (Google and I didn't find it)
> >
> > Thanks in advance.
> > Rick
> >
>
Re: Getting started developing/debugging Nifi processors
Posted by Matt Gilman <ma...@gmail.com>.
Rick,
You should be able to run NiFi from where you installed it from the
.tar.gz. Once you've enabled remote debugging (via the line in the
bootstrap.conf) you'll be able to attach the remote debugger in your IDE.
The remote debugger will need to be configured with the port, though I
believe 8000 is the default. The only other nugget here is the suspend
flag. By default it's set to 'n', but you can change to 'y' if you need to
debug something that is in your processors initialization logic. It will
suspend launching until the remote debugger is attached.
Matt
On Sun, Aug 30, 2015 at 3:41 PM, Rick Braddy <rb...@softnas.com> wrote:
> Joe,
>
> Thanks for that information. I will start with NetBeans, as I've used it
> before on some other projects.
>
> On #2, during development and debugging are you running Nifi within the
> source tree or in a separate target tree where it's been installed from the
> .tar.gz?
>
> Rick
>
> -----Original Message-----
> From: Joe Witt [mailto:joe.witt@gmail.com]
> Sent: Sunday, August 30, 2015 10:43 AM
> To: dev@nifi.apache.org
> Subject: Re: Getting started developing/debugging Nifi processors
>
> Rick
>
> These are great and should be turned into a FAQ for developers so they can
> easily get information on these. Some aspects of these are covered in the
> developer guide but not clearly or completely.
>
> #1 I know folks on the team that use Eclipse, IntelliJ, and Netbeans and
> probably in that order of popularity. I personally prefer Netbeans because
> the integration with Maven just flat out works - flawlessly. That is
> *definitely* a personal preference thing though.
>
> #2 You can really easily bind to a running NiFi instance or cause binding
> to occur as soon as NiFi starts up. This is so you can truly debug a
> running instance including stepping through code. In the
> conf/boostrap.conf file check these two lines out:
>
> # Enable Remote Debugging
>
> #java.arg.debug=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000
>
> By uncommenting that second line you can attach a debugger at runtime to
> port 8000 and step through the code. Super cool.
>
> #3 By far the most efficient way to do this is through effective unit
> testing. The mock framework makes this work out quite well. Now, it is
> challenging at times to effectively unit test things which interact with
> external services. For those cases where you truly must update the version
> of nars in an actual instance of NiFi you may want to just script out
> commands to copy over the old nar and restart nifi.
>
> Thanks
> Joe
>
> On Sun, Aug 30, 2015 at 1:34 PM, Rick Braddy <rb...@softnas.com> wrote:
> > Hi,
> >
> > The Nifi Developer's Guide is a great read with lots of useful
> information. However, as a relative noob to Nifi development, I still have
> a few questions that seemed unanswered (or perhaps I overlooked):
> >
> >
> > 1. Recommended IDE for Nifi - given Nifi development involves both
> Java and Maven, is there a recommended or preferred IDE (e.g., NetBeans vs.
> Eclipse - other?) that's considered better? I realize this is also
> probably personal preference, but would like to know what the core Nifi
> team uses and recommends for debugging, both local and remote Nifi
> instances.
> >
> >
> > 2. Debugging mode - given Nifi often runs as a Linux service, for
> debugging are you typically attaching to one or more running processes or
> launching Nifi separately to debug it? If attaching, what does one attach
> to, given many running java processes come and go?
> >
> >
> > 3. Processor development vs. NAR packaging - when developing and
> debugging, given the run-time Nifi is installed in a different directory
> tree vs. the source tree, what is the most productive approach to inline
> development and debugging (vs. making changes, packaging and then
> installing NAR package into run-time area?
> >
> > Is there any documentation or blog posts covering the above? (Google
> > and I didn't find it)
> >
> > Thanks in advance.
> > Rick
> >
>
RE: Getting started developing/debugging Nifi processors
Posted by Rick Braddy <rb...@softnas.com>.
Joe,
Thanks for that information. I will start with NetBeans, as I've used it before on some other projects.
On #2, during development and debugging are you running Nifi within the source tree or in a separate target tree where it's been installed from the .tar.gz?
Rick
-----Original Message-----
From: Joe Witt [mailto:joe.witt@gmail.com]
Sent: Sunday, August 30, 2015 10:43 AM
To: dev@nifi.apache.org
Subject: Re: Getting started developing/debugging Nifi processors
Rick
These are great and should be turned into a FAQ for developers so they can easily get information on these. Some aspects of these are covered in the developer guide but not clearly or completely.
#1 I know folks on the team that use Eclipse, IntelliJ, and Netbeans and probably in that order of popularity. I personally prefer Netbeans because the integration with Maven just flat out works - flawlessly. That is *definitely* a personal preference thing though.
#2 You can really easily bind to a running NiFi instance or cause binding to occur as soon as NiFi starts up. This is so you can truly debug a running instance including stepping through code. In the conf/boostrap.conf file check these two lines out:
# Enable Remote Debugging
#java.arg.debug=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000
By uncommenting that second line you can attach a debugger at runtime to port 8000 and step through the code. Super cool.
#3 By far the most efficient way to do this is through effective unit testing. The mock framework makes this work out quite well. Now, it is challenging at times to effectively unit test things which interact with external services. For those cases where you truly must update the version of nars in an actual instance of NiFi you may want to just script out commands to copy over the old nar and restart nifi.
Thanks
Joe
On Sun, Aug 30, 2015 at 1:34 PM, Rick Braddy <rb...@softnas.com> wrote:
> Hi,
>
> The Nifi Developer's Guide is a great read with lots of useful information. However, as a relative noob to Nifi development, I still have a few questions that seemed unanswered (or perhaps I overlooked):
>
>
> 1. Recommended IDE for Nifi - given Nifi development involves both Java and Maven, is there a recommended or preferred IDE (e.g., NetBeans vs. Eclipse - other?) that's considered better? I realize this is also probably personal preference, but would like to know what the core Nifi team uses and recommends for debugging, both local and remote Nifi instances.
>
>
> 2. Debugging mode - given Nifi often runs as a Linux service, for debugging are you typically attaching to one or more running processes or launching Nifi separately to debug it? If attaching, what does one attach to, given many running java processes come and go?
>
>
> 3. Processor development vs. NAR packaging - when developing and debugging, given the run-time Nifi is installed in a different directory tree vs. the source tree, what is the most productive approach to inline development and debugging (vs. making changes, packaging and then installing NAR package into run-time area?
>
> Is there any documentation or blog posts covering the above? (Google
> and I didn't find it)
>
> Thanks in advance.
> Rick
>
Re: Getting started developing/debugging Nifi processors
Posted by Joe Witt <jo...@gmail.com>.
Rick
These are great and should be turned into a FAQ for developers so they
can easily get information on these. Some aspects of these are
covered in the developer guide but not clearly or completely.
#1 I know folks on the team that use Eclipse, IntelliJ, and Netbeans
and probably in that order of popularity. I personally prefer
Netbeans because the integration with Maven just flat out works -
flawlessly. That is *definitely* a personal preference thing though.
#2 You can really easily bind to a running NiFi instance or cause
binding to occur as soon as NiFi starts up. This is so you can truly
debug a running instance including stepping through code. In the
conf/boostrap.conf file check these two lines out:
# Enable Remote Debugging
#java.arg.debug=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000
By uncommenting that second line you can attach a debugger at runtime
to port 8000 and step through the code. Super cool.
#3 By far the most efficient way to do this is through effective unit
testing. The mock framework makes this work out quite well. Now, it
is challenging at times to effectively unit test things which interact
with external services. For those cases where you truly must update
the version of nars in an actual instance of NiFi you may want to just
script out commands to copy over the old nar and restart nifi.
Thanks
Joe
On Sun, Aug 30, 2015 at 1:34 PM, Rick Braddy <rb...@softnas.com> wrote:
> Hi,
>
> The Nifi Developer's Guide is a great read with lots of useful information. However, as a relative noob to Nifi development, I still have a few questions that seemed unanswered (or perhaps I overlooked):
>
>
> 1. Recommended IDE for Nifi - given Nifi development involves both Java and Maven, is there a recommended or preferred IDE (e.g., NetBeans vs. Eclipse - other?) that's considered better? I realize this is also probably personal preference, but would like to know what the core Nifi team uses and recommends for debugging, both local and remote Nifi instances.
>
>
> 2. Debugging mode - given Nifi often runs as a Linux service, for debugging are you typically attaching to one or more running processes or launching Nifi separately to debug it? If attaching, what does one attach to, given many running java processes come and go?
>
>
> 3. Processor development vs. NAR packaging - when developing and debugging, given the run-time Nifi is installed in a different directory tree vs. the source tree, what is the most productive approach to inline development and debugging (vs. making changes, packaging and then installing NAR package into run-time area?
>
> Is there any documentation or blog posts covering the above? (Google and I didn't find it)
>
> Thanks in advance.
> Rick
>