You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Russell Bateman <ru...@windofkeltia.com> on 2016/12/14 18:05:41 UTC
Adding JVM arguments when running NiFi...
I've doctored /conf/bootstrap.conf/ to contain these additional lines:
java.arg.15=-XX:+UnlockCommercialFeatures
java.arg.16=-XX:+FlightRecorder
java.arg.17=-XX:StartFlightRecording=duration=120m,filename=recording.jfr
In the end, NiFi's grumpy about this and won't start (from
/logs/nifi-bootstrap.log/):
2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
org.apache.nifi.StdErr Error: *To use 'FlightRecorder', first unlock
using -XX:+UnlockCommercialFeatures.*
2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
org.apache.nifi.StdErr Error: Could not create the Java Virtual Machine.
2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
org.apache.nifi.StdErr Error: A fatal exception has occurred. Program
will exit.
2016-12-14 10:39:36,507 INFO [main] org.apache.nifi.bootstrap.RunNiFi
NiFi never started. Will not restart NiFi
I tried using all options as one (in case the order is disturbed, which
it was):
java.arg.15=-XX:+UnlockCommercialFeatures-XX:+FlightRecorder-XX:StartFlightRecording=duration=120m,filename=recording.jfr
and then I get:
*2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
org.apache.nifi.StdErr Unrecognized VM option 'UnlockCommercialFeatures
-XX:+FlightRecorder
-XX:StartFlightRecording=duration=120m,filename=recording.jfr'*
2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
org.apache.nifi.StdErr Error: Could not create the Java Virtual Machine.
2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
org.apache.nifi.StdErr Error: A fatal exception has occurred. Program
will exit.
2016-12-14 10:50:07,598 INFO [main] org.apache.nifi.bootstrap.RunNiFi
NiFi never started. Will not restart NiFi
Here's the second command line from /logs/nifi-bootstrap.log/, which
I've wrapped for convenience in reading (see JVM version here too):
2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command
Working Directory: /home/russ/dev/nifi/nifi-1.1.0
2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command
Command: */home/russ/dev/jdk1.8.0_112/bin/java*
-classpath
/home/russ/dev/nifi/nifi-1.1.0/./conf:/home/russ/dev/nifi/nifi-1.1.0/./lib/jcl-over-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/logback-core-1.1.3.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-documentation-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-nar-utils-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/logback-classic-1.1.3.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/jul-to-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-runtime-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/slf4j-api-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-api-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-framework-api-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/log4j-over-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-properties-1.1.0.jar
-Dorg.apache.jasper.compiler.disablejsr199=true
-Xmx512m
-Xms512m
-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=9191
*-XX:+UnlockCommercialFeatures
-XX:+FlightRecorder
-XX:StartFlightRecording=duration=120m,filename=recording.jfr*
-Dsun.net.http.allowRestrictedHeaders=true
-Djava.net.preferIPv4Stack=true
-Djava.awt.headless=true
-XX:+UseG1GC
-Djava.protocol.handler.pkgs=sun.net.www.protocol
-Dnifi.properties.file.path=/home/russ/dev/nifi/nifi-1.1.0/./conf/nifi.properties
-Dnifi.bootstrap.listen.port=41736
-Dapp=NiFi
-Dorg.apache.nifi.bootstrap.config.log.dir=/home/russ/dev/nifi/nifi-1.1.0/logs
org.apache.nifi.NiFi
Last, I frantically tried this:
java.arg.15="-XX:+UnlockCommercialFeatures -XX:+FlightRecorder
-XX:StartFlightRecording=duration=120m,filename=recording.jfr"
and then got:
*2016-12-14 11:01:36,659 ERROR [NiFi logging handler]
org.apache.nifi.StdErr Error: Could not find or load main class
"-XX:+UnlockCommercialFeatures -XX:+FlightRecorder
-XX:StartFlightRecording=duration=120m,filename=recording.jfr"*
What's the canonical way of adding/inserting my own JVM commands?
Alternatively, is NiFi analyzing and blocking -XX:arguments to the JVM
as illegal or unrecognized?
Thanks,
Russ
Re: Adding JVM arguments when running NiFi...
Posted by Andrew Grande <ap...@gmail.com>.
Sounds like switching to the LinkedHashMap could address.the JVM startup
flags ordering g?
Andrew
On Thu, Dec 15, 2016, 1:11 PM Russell Bateman <ru...@windofkeltia.com> wrote:
> Joe,
>
> I have more detail and success to report...
>
> There is no problem in NiFi /per se/. It was not really necessary to dig
> down into the code really because there's no problem there. It's just
> how java.arg./n/is consumed that creates trouble. One must play around
> with the value of /n/ when argument ordering is crucial. In the present
> case, I was able to turn on JFR using the following (in bold) in
> /conf/bootstrap.conf/:
>
> #Set headless mode by default
> java.arg.14=-Djava.awt.headless=true
>
> *# Light up the Java Flight Recorder...**
> **java.arg.32=-XX:+UnlockCommercialFeatures**
> **java.arg.31=-XX:+FlightRecorder**
>
> **java.arg.42=-XX:StartFlightRecording=duration=120m,filename=recording.jfr*
>
> # Master key in hexadecimal format for encrypted sensitive
> configuration values
> nifi.bootstrap.sensitive.key=
>
> The point was that -XX:+UnlockCommercialFeatures must absolutely precede
> the other two options. Because of these arguments passing through
> HashMap, some values of /n/ will fail to sort an argument before others
> and the addition of other arguments might also come along and disturb
> this order. What I did here was by trial and error. I guess NiFi code
> could man-handle /n/ in every case such that the order be respected. I'm
> guessing that my case is the only or one of the very rare times options
> are order-dependent. I wouldn't personally up-vote a JIRA to fix this.
>
> Hoping this forum has an adequate profile out there in Googleland, this
> very discussion will be enough to help the next guy.
>
> Russ
>
> P.S. My trial and error went pretty fast using this command line between
> reassigning instances of /n/ for these arguments. It took me maybe 6
> tries. (Script /bounce-nifi.sh/ does little more than shut down, then
> start up NiFi, something I do a hundred times per day across at least
> two versions.)
>
> ~/dev/nifi/nifi-1.1.0/logs $ rm *.log ; bounce-nifi.sh 1.1.0 ; tail
> -f nifi-bootstrap.log
>
>
>
>
> On 12/14/2016 07:41 PM, Joe Witt wrote:
> > The way you set it up initially is what I'd thought would have done
> > the trick. Perhaps we're not ordering the arguments in the same
> > manner supplied. Will need to look into that.
> >
> > Thanks
> > Joe
> >
> > On Wed, Dec 14, 2016 at 9:00 PM, Russell Bateman <ru...@windofkeltia.com>
> wrote:
> >> Of course, as many have done, I've run Java applications with JFR
> enabled
> >> using these options against this very JVM (jdk1.8.0_112). So, it isn't a
> >> problem for the JVM I'm using. I haven't finished digging down into
> >> ProcessBuilder, or deeper, to figure out why these options are not
> getting
> >> love. I'll get back at it tomorrow and report back.
> >>
> >>
> >> On 12/14/2016 11:05 AM, Russell Bateman wrote:
> >>>
> >>> I've doctored /conf/bootstrap.conf/ to contain these additional lines:
> >>>
> >>> java.arg.15=-XX:+UnlockCommercialFeatures
> >>> java.arg.16=-XX:+FlightRecorder
> >>>
> >>>
> java.arg.17=-XX:StartFlightRecording=duration=120m,filename=recording.jfr
> >>>
> >>>
> >>> In the end, NiFi's grumpy about this and won't start (from
> >>> /logs/nifi-bootstrap.log/):
> >>>
> >>> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
> >>> org.apache.nifi.StdErr Error: *To use 'FlightRecorder', first unlock
> using
> >>> -XX:+UnlockCommercialFeatures.*
> >>> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
> >>> org.apache.nifi.StdErr Error: Could not create the Java Virtual
> Machine.
> >>> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
> >>> org.apache.nifi.StdErr Error: A fatal exception has occurred. Program
> will
> >>> exit.
> >>> 2016-12-14 10:39:36,507 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> NiFi
> >>> never started. Will not restart NiFi
> >>>
> >>> I tried using all options as one (in case the order is disturbed,
> which it
> >>> was):
> >>>
> >>>
> >>>
> java.arg.15=-XX:+UnlockCommercialFeatures-XX:+FlightRecorder-XX:StartFlightRecording=duration=120m,filename=recording.jfr
> >>>
> >>>
> >>> and then I get:
> >>>
> >>> *2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
> >>> org.apache.nifi.StdErr Unrecognized VM option 'UnlockCommercialFeatures
> >>> -XX:+FlightRecorder
> >>> -XX:StartFlightRecording=duration=120m,filename=recording.jfr'*
> >>> 2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
> >>> org.apache.nifi.StdErr Error: Could not create the Java Virtual
> Machine.
> >>> 2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
> >>> org.apache.nifi.StdErr Error: A fatal exception has occurred. Program
> will
> >>> exit.
> >>> 2016-12-14 10:50:07,598 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> NiFi
> >>> never started. Will not restart NiFi
> >>>
> >>> Here's the second command line from /logs/nifi-bootstrap.log/, which
> I've
> >>> wrapped for convenience in reading (see JVM version here too):
> >>>
> >>> 2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command
> >>> Working Directory: /home/russ/dev/nifi/nifi-1.1.0
> >>> 2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command
> >>> Command: */home/russ/dev/jdk1.8.0_112/bin/java*
> >>> -classpath
> >>>
> /home/russ/dev/nifi/nifi-1.1.0/./conf:/home/russ/dev/nifi/nifi-1.1.0/./lib/jcl-over-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/logback-core-1.1.3.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-documentation-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-nar-utils-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/logback-classic-1.1.3.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/jul-to-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-runtime-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/slf4j-api-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-api-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-framework-api-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/log4j-over-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-properties-1.1.0.jar
> >>> -Dorg.apache.jasper.compiler.disablejsr199=true
> >>> -Xmx512m
> >>> -Xms512m
> >>> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=9191
> >>> *-XX:+UnlockCommercialFeatures
> >>> -XX:+FlightRecorder
> >>> -XX:StartFlightRecording=duration=120m,filename=recording.jfr*
> >>> -Dsun.net.http.allowRestrictedHeaders=true
> >>> -Djava.net.preferIPv4Stack=true
> >>> -Djava.awt.headless=true
> >>> -XX:+UseG1GC
> >>> -Djava.protocol.handler.pkgs=sun.net.www.protocol
> >>>
> >>>
> -Dnifi.properties.file.path=/home/russ/dev/nifi/nifi-1.1.0/./conf/nifi.properties
> >>> -Dnifi.bootstrap.listen.port=41736
> >>> -Dapp=NiFi
> >>>
> >>>
> -Dorg.apache.nifi.bootstrap.config.log.dir=/home/russ/dev/nifi/nifi-1.1.0/logs
> >>> org.apache.nifi.NiFi
> >>>
> >>> Last, I frantically tried this:
> >>>
> >>> java.arg.15="-XX:+UnlockCommercialFeatures -XX:+FlightRecorder
> >>> -XX:StartFlightRecording=duration=120m,filename=recording.jfr"
> >>>
> >>>
> >>> and then got:
> >>>
> >>> *2016-12-14 11:01:36,659 ERROR [NiFi logging handler]
> >>> org.apache.nifi.StdErr Error: Could not find or load main class
> >>> "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder
> >>> -XX:StartFlightRecording=duration=120m,filename=recording.jfr"*
> >>>
> >>>
> >>> What's the canonical way of adding/inserting my own JVM commands?
> >>> Alternatively, is NiFi analyzing and blocking -XX:arguments to the JVM
> as
> >>> illegal or unrecognized?
> >>>
> >>> Thanks,
> >>>
> >>> Russ
> >>
>
>
Re: Adding JVM arguments when running NiFi...
Posted by Russell Bateman <ru...@windofkeltia.com>.
Joe,
I have more detail and success to report...
There is no problem in NiFi /per se/. It was not really necessary to dig
down into the code really because there's no problem there. It's just
how java.arg./n/is consumed that creates trouble. One must play around
with the value of /n/ when argument ordering is crucial. In the present
case, I was able to turn on JFR using the following (in bold) in
/conf/bootstrap.conf/:
#Set headless mode by default
java.arg.14=-Djava.awt.headless=true
*# Light up the Java Flight Recorder...**
**java.arg.32=-XX:+UnlockCommercialFeatures**
**java.arg.31=-XX:+FlightRecorder**
**java.arg.42=-XX:StartFlightRecording=duration=120m,filename=recording.jfr*
# Master key in hexadecimal format for encrypted sensitive
configuration values
nifi.bootstrap.sensitive.key=
The point was that -XX:+UnlockCommercialFeatures must absolutely precede
the other two options. Because of these arguments passing through
HashMap, some values of /n/ will fail to sort an argument before others
and the addition of other arguments might also come along and disturb
this order. What I did here was by trial and error. I guess NiFi code
could man-handle /n/ in every case such that the order be respected. I'm
guessing that my case is the only or one of the very rare times options
are order-dependent. I wouldn't personally up-vote a JIRA to fix this.
Hoping this forum has an adequate profile out there in Googleland, this
very discussion will be enough to help the next guy.
Russ
P.S. My trial and error went pretty fast using this command line between
reassigning instances of /n/ for these arguments. It took me maybe 6
tries. (Script /bounce-nifi.sh/ does little more than shut down, then
start up NiFi, something I do a hundred times per day across at least
two versions.)
~/dev/nifi/nifi-1.1.0/logs $ rm *.log ; bounce-nifi.sh 1.1.0 ; tail
-f nifi-bootstrap.log
On 12/14/2016 07:41 PM, Joe Witt wrote:
> The way you set it up initially is what I'd thought would have done
> the trick. Perhaps we're not ordering the arguments in the same
> manner supplied. Will need to look into that.
>
> Thanks
> Joe
>
> On Wed, Dec 14, 2016 at 9:00 PM, Russell Bateman <ru...@windofkeltia.com> wrote:
>> Of course, as many have done, I've run Java applications with JFR enabled
>> using these options against this very JVM (jdk1.8.0_112). So, it isn't a
>> problem for the JVM I'm using. I haven't finished digging down into
>> ProcessBuilder, or deeper, to figure out why these options are not getting
>> love. I'll get back at it tomorrow and report back.
>>
>>
>> On 12/14/2016 11:05 AM, Russell Bateman wrote:
>>>
>>> I've doctored /conf/bootstrap.conf/ to contain these additional lines:
>>>
>>> java.arg.15=-XX:+UnlockCommercialFeatures
>>> java.arg.16=-XX:+FlightRecorder
>>>
>>> java.arg.17=-XX:StartFlightRecording=duration=120m,filename=recording.jfr
>>>
>>>
>>> In the end, NiFi's grumpy about this and won't start (from
>>> /logs/nifi-bootstrap.log/):
>>>
>>> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
>>> org.apache.nifi.StdErr Error: *To use 'FlightRecorder', first unlock using
>>> -XX:+UnlockCommercialFeatures.*
>>> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
>>> org.apache.nifi.StdErr Error: Could not create the Java Virtual Machine.
>>> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
>>> org.apache.nifi.StdErr Error: A fatal exception has occurred. Program will
>>> exit.
>>> 2016-12-14 10:39:36,507 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi
>>> never started. Will not restart NiFi
>>>
>>> I tried using all options as one (in case the order is disturbed, which it
>>> was):
>>>
>>>
>>> java.arg.15=-XX:+UnlockCommercialFeatures-XX:+FlightRecorder-XX:StartFlightRecording=duration=120m,filename=recording.jfr
>>>
>>>
>>> and then I get:
>>>
>>> *2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
>>> org.apache.nifi.StdErr Unrecognized VM option 'UnlockCommercialFeatures
>>> -XX:+FlightRecorder
>>> -XX:StartFlightRecording=duration=120m,filename=recording.jfr'*
>>> 2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
>>> org.apache.nifi.StdErr Error: Could not create the Java Virtual Machine.
>>> 2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
>>> org.apache.nifi.StdErr Error: A fatal exception has occurred. Program will
>>> exit.
>>> 2016-12-14 10:50:07,598 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi
>>> never started. Will not restart NiFi
>>>
>>> Here's the second command line from /logs/nifi-bootstrap.log/, which I've
>>> wrapped for convenience in reading (see JVM version here too):
>>>
>>> 2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command
>>> Working Directory: /home/russ/dev/nifi/nifi-1.1.0
>>> 2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command
>>> Command: */home/russ/dev/jdk1.8.0_112/bin/java*
>>> -classpath
>>> /home/russ/dev/nifi/nifi-1.1.0/./conf:/home/russ/dev/nifi/nifi-1.1.0/./lib/jcl-over-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/logback-core-1.1.3.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-documentation-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-nar-utils-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/logback-classic-1.1.3.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/jul-to-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-runtime-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/slf4j-api-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-api-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-framework-api-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/log4j-over-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-properties-1.1.0.jar
>>> -Dorg.apache.jasper.compiler.disablejsr199=true
>>> -Xmx512m
>>> -Xms512m
>>> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=9191
>>> *-XX:+UnlockCommercialFeatures
>>> -XX:+FlightRecorder
>>> -XX:StartFlightRecording=duration=120m,filename=recording.jfr*
>>> -Dsun.net.http.allowRestrictedHeaders=true
>>> -Djava.net.preferIPv4Stack=true
>>> -Djava.awt.headless=true
>>> -XX:+UseG1GC
>>> -Djava.protocol.handler.pkgs=sun.net.www.protocol
>>>
>>> -Dnifi.properties.file.path=/home/russ/dev/nifi/nifi-1.1.0/./conf/nifi.properties
>>> -Dnifi.bootstrap.listen.port=41736
>>> -Dapp=NiFi
>>>
>>> -Dorg.apache.nifi.bootstrap.config.log.dir=/home/russ/dev/nifi/nifi-1.1.0/logs
>>> org.apache.nifi.NiFi
>>>
>>> Last, I frantically tried this:
>>>
>>> java.arg.15="-XX:+UnlockCommercialFeatures -XX:+FlightRecorder
>>> -XX:StartFlightRecording=duration=120m,filename=recording.jfr"
>>>
>>>
>>> and then got:
>>>
>>> *2016-12-14 11:01:36,659 ERROR [NiFi logging handler]
>>> org.apache.nifi.StdErr Error: Could not find or load main class
>>> "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder
>>> -XX:StartFlightRecording=duration=120m,filename=recording.jfr"*
>>>
>>>
>>> What's the canonical way of adding/inserting my own JVM commands?
>>> Alternatively, is NiFi analyzing and blocking -XX:arguments to the JVM as
>>> illegal or unrecognized?
>>>
>>> Thanks,
>>>
>>> Russ
>>
Re: Adding JVM arguments when running NiFi...
Posted by Joe Witt <jo...@gmail.com>.
The way you set it up initially is what I'd thought would have done
the trick. Perhaps we're not ordering the arguments in the same
manner supplied. Will need to look into that.
Thanks
Joe
On Wed, Dec 14, 2016 at 9:00 PM, Russell Bateman <ru...@windofkeltia.com> wrote:
> Of course, as many have done, I've run Java applications with JFR enabled
> using these options against this very JVM (jdk1.8.0_112). So, it isn't a
> problem for the JVM I'm using. I haven't finished digging down into
> ProcessBuilder, or deeper, to figure out why these options are not getting
> love. I'll get back at it tomorrow and report back.
>
>
> On 12/14/2016 11:05 AM, Russell Bateman wrote:
>>
>>
>> I've doctored /conf/bootstrap.conf/ to contain these additional lines:
>>
>> java.arg.15=-XX:+UnlockCommercialFeatures
>> java.arg.16=-XX:+FlightRecorder
>>
>> java.arg.17=-XX:StartFlightRecording=duration=120m,filename=recording.jfr
>>
>>
>> In the end, NiFi's grumpy about this and won't start (from
>> /logs/nifi-bootstrap.log/):
>>
>> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
>> org.apache.nifi.StdErr Error: *To use 'FlightRecorder', first unlock using
>> -XX:+UnlockCommercialFeatures.*
>> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
>> org.apache.nifi.StdErr Error: Could not create the Java Virtual Machine.
>> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
>> org.apache.nifi.StdErr Error: A fatal exception has occurred. Program will
>> exit.
>> 2016-12-14 10:39:36,507 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi
>> never started. Will not restart NiFi
>>
>> I tried using all options as one (in case the order is disturbed, which it
>> was):
>>
>>
>> java.arg.15=-XX:+UnlockCommercialFeatures-XX:+FlightRecorder-XX:StartFlightRecording=duration=120m,filename=recording.jfr
>>
>>
>> and then I get:
>>
>> *2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
>> org.apache.nifi.StdErr Unrecognized VM option 'UnlockCommercialFeatures
>> -XX:+FlightRecorder
>> -XX:StartFlightRecording=duration=120m,filename=recording.jfr'*
>> 2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
>> org.apache.nifi.StdErr Error: Could not create the Java Virtual Machine.
>> 2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
>> org.apache.nifi.StdErr Error: A fatal exception has occurred. Program will
>> exit.
>> 2016-12-14 10:50:07,598 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi
>> never started. Will not restart NiFi
>>
>> Here's the second command line from /logs/nifi-bootstrap.log/, which I've
>> wrapped for convenience in reading (see JVM version here too):
>>
>> 2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command
>> Working Directory: /home/russ/dev/nifi/nifi-1.1.0
>> 2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command
>> Command: */home/russ/dev/jdk1.8.0_112/bin/java*
>> -classpath
>> /home/russ/dev/nifi/nifi-1.1.0/./conf:/home/russ/dev/nifi/nifi-1.1.0/./lib/jcl-over-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/logback-core-1.1.3.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-documentation-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-nar-utils-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/logback-classic-1.1.3.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/jul-to-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-runtime-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/slf4j-api-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-api-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-framework-api-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/log4j-over-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-properties-1.1.0.jar
>> -Dorg.apache.jasper.compiler.disablejsr199=true
>> -Xmx512m
>> -Xms512m
>> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=9191
>> *-XX:+UnlockCommercialFeatures
>> -XX:+FlightRecorder
>> -XX:StartFlightRecording=duration=120m,filename=recording.jfr*
>> -Dsun.net.http.allowRestrictedHeaders=true
>> -Djava.net.preferIPv4Stack=true
>> -Djava.awt.headless=true
>> -XX:+UseG1GC
>> -Djava.protocol.handler.pkgs=sun.net.www.protocol
>>
>> -Dnifi.properties.file.path=/home/russ/dev/nifi/nifi-1.1.0/./conf/nifi.properties
>> -Dnifi.bootstrap.listen.port=41736
>> -Dapp=NiFi
>>
>> -Dorg.apache.nifi.bootstrap.config.log.dir=/home/russ/dev/nifi/nifi-1.1.0/logs
>> org.apache.nifi.NiFi
>>
>> Last, I frantically tried this:
>>
>> java.arg.15="-XX:+UnlockCommercialFeatures -XX:+FlightRecorder
>> -XX:StartFlightRecording=duration=120m,filename=recording.jfr"
>>
>>
>> and then got:
>>
>> *2016-12-14 11:01:36,659 ERROR [NiFi logging handler]
>> org.apache.nifi.StdErr Error: Could not find or load main class
>> "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder
>> -XX:StartFlightRecording=duration=120m,filename=recording.jfr"*
>>
>>
>> What's the canonical way of adding/inserting my own JVM commands?
>> Alternatively, is NiFi analyzing and blocking -XX:arguments to the JVM as
>> illegal or unrecognized?
>>
>> Thanks,
>>
>> Russ
>
>
Re: Adding JVM arguments when running NiFi...
Posted by Russell Bateman <ru...@windofkeltia.com>.
Of course, as many have done, I've run Java applications with JFR
enabled using these options against this very JVM (jdk1.8.0_112). So, it
isn't a problem for the JVM I'm using. I haven't finished digging down
into ProcessBuilder, or deeper, to figure out why these options are not
getting love. I'll get back at it tomorrow and report back.
On 12/14/2016 11:05 AM, Russell Bateman wrote:
>
> I've doctored /conf/bootstrap.conf/ to contain these additional lines:
>
> java.arg.15=-XX:+UnlockCommercialFeatures
> java.arg.16=-XX:+FlightRecorder
> java.arg.17=-XX:StartFlightRecording=duration=120m,filename=recording.jfr
>
>
> In the end, NiFi's grumpy about this and won't start (from
> /logs/nifi-bootstrap.log/):
>
> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
> org.apache.nifi.StdErr Error: *To use 'FlightRecorder', first unlock
> using -XX:+UnlockCommercialFeatures.*
> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
> org.apache.nifi.StdErr Error: Could not create the Java Virtual Machine.
> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
> org.apache.nifi.StdErr Error: A fatal exception has occurred. Program
> will exit.
> 2016-12-14 10:39:36,507 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> NiFi never started. Will not restart NiFi
>
> I tried using all options as one (in case the order is disturbed,
> which it was):
>
> java.arg.15=-XX:+UnlockCommercialFeatures-XX:+FlightRecorder-XX:StartFlightRecording=duration=120m,filename=recording.jfr
>
>
> and then I get:
>
> *2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
> org.apache.nifi.StdErr Unrecognized VM option
> 'UnlockCommercialFeatures -XX:+FlightRecorder
> -XX:StartFlightRecording=duration=120m,filename=recording.jfr'*
> 2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
> org.apache.nifi.StdErr Error: Could not create the Java Virtual Machine.
> 2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
> org.apache.nifi.StdErr Error: A fatal exception has occurred. Program
> will exit.
> 2016-12-14 10:50:07,598 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> NiFi never started. Will not restart NiFi
>
> Here's the second command line from /logs/nifi-bootstrap.log/, which
> I've wrapped for convenience in reading (see JVM version here too):
>
> 2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command
> Working Directory: /home/russ/dev/nifi/nifi-1.1.0
> 2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command
> Command: */home/russ/dev/jdk1.8.0_112/bin/java*
> -classpath
> /home/russ/dev/nifi/nifi-1.1.0/./conf:/home/russ/dev/nifi/nifi-1.1.0/./lib/jcl-over-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/logback-core-1.1.3.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-documentation-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-nar-utils-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/logback-classic-1.1.3.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/jul-to-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-runtime-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/slf4j-api-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-api-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-framework-api-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/log4j-over-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-properties-1.1.0.jar
> -Dorg.apache.jasper.compiler.disablejsr199=true
> -Xmx512m
> -Xms512m
> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=9191
> *-XX:+UnlockCommercialFeatures
> -XX:+FlightRecorder
> -XX:StartFlightRecording=duration=120m,filename=recording.jfr*
> -Dsun.net.http.allowRestrictedHeaders=true
> -Djava.net.preferIPv4Stack=true
> -Djava.awt.headless=true
> -XX:+UseG1GC
> -Djava.protocol.handler.pkgs=sun.net.www.protocol
> -Dnifi.properties.file.path=/home/russ/dev/nifi/nifi-1.1.0/./conf/nifi.properties
> -Dnifi.bootstrap.listen.port=41736
> -Dapp=NiFi
> -Dorg.apache.nifi.bootstrap.config.log.dir=/home/russ/dev/nifi/nifi-1.1.0/logs
> org.apache.nifi.NiFi
>
> Last, I frantically tried this:
>
> java.arg.15="-XX:+UnlockCommercialFeatures -XX:+FlightRecorder
> -XX:StartFlightRecording=duration=120m,filename=recording.jfr"
>
>
> and then got:
>
> *2016-12-14 11:01:36,659 ERROR [NiFi logging handler]
> org.apache.nifi.StdErr Error: Could not find or load main class
> "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder
> -XX:StartFlightRecording=duration=120m,filename=recording.jfr"*
>
>
> What's the canonical way of adding/inserting my own JVM commands?
> Alternatively, is NiFi analyzing and blocking -XX:arguments to the JVM
> as illegal or unrecognized?
>
> Thanks,
>
> Russ
Re: Adding JVM arguments when running NiFi...
Posted by Russell Bateman <ru...@windofkeltia.com>.
Thanks for pointing that out, Mark.
The net result of what I wrote is just that: numeric arguments in order
of N, then whatever else is there, unsorted. I did nothing, however, to
handle conflicting values of N. I don't claim the code I attached to the
JIRA to be the best solution, only an illustration of a solution. In my
opinion, except for respecting the value of N, this is all using a 3lb
hammer to crack a peanut, but it has to be done and it's up to the real
committers to decide how far to run with this. Personally, I'd just use
the code I wrote, with added handling for duplicate values of N, then
add the non-numeric arguments and call it good.
For most folk, it will never make any difference.
Russ
P.S. I'm promising a wiki-expos on how to use the Java Flight Recorder
and Java Mission Control as soon as I gain authorization and someone
explains to me how to create the page in the wiki.
On 01/18/2017 08:31 AM, Mark Payne wrote:
> Russ,
>
> Sorry - I didn't see this sooner. That is accurate, that 'N' can be non-numeric. By default, in bootstrap.conf, there is actually a property named "java.arg.debug=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000" but it is commented out.
>
> I think that if we want to guarantee ordering, we need to do some sort of sorting on the keys. For example, first comes all numbers in numeric ordering (i.e., - not lexigraphical ordering - so java.arg.1, java.arg.2, java.arg.3, ..., java.arg.10, java.arg.11, ...), followed by non-numeric values in lexigraphical ordering. This would make it much easier to guarantee ordering by simply using numbers but doesn't enforce numbers to be used. We didn't enforce numbers to be used because historically we've found that people are inclined to always use a one-up number. So when they want to add a new property, they go searching through the file looking for the latest number used and then use the next number; then, if they un-comment a line, that one may conflict resulting in two arguments with the same numbers and one of them is overridden.
>
> Thanks
> -Mark
>
>
>> On Jan 18, 2017, at 10:22 AM, Russell Bateman <ru...@windofkeltia.com> wrote:
>>
>> For my own purposes, I fixed this in 0.7.1 /nifi-bootstrap///src/main/java/org/////apache/nifi/bootstrap/RunNiFi.java/, but at Joe's suggestion, I created JIRA NIFI-3364 hoping some committer will get to it for all code branches-forward. The solution I used was made to be as little disruptive to the existing code as possible. To that JIRA I attached the solution pinched between comments in a larger test demonstrating it.
>>
>> I had found that I could trick the order in my local environment by trial and error, however, moving to another host and with the additional need to dominate the order of JMX remote definitions as well (in order to reach NiFi remote from Java Mission Control), it just became impossible to achieve it.
>>
>> The code I wrote isn't the only way to fix nor even necessarily the best, but it works supporting both the ordering of /N/in java.arg.Nas well as (did you know this?) the use of non-numeric options for the position of /N/.
>>
>> Russ
>>
>> On 12/14/2016 11:05 AM, Russell Bateman wrote:
>>> I've doctored /conf/bootstrap.conf/ to contain these additional lines:
>>>
>>> java.arg.15=-XX:+UnlockCommercialFeatures
>>> java.arg.16=-XX:+FlightRecorder
>>> java.arg.17=-XX:StartFlightRecording=duration=120m,filename=recording.jfr
>>>
>>>
>>> In the end, NiFi's grumpy about this and won't start (from /logs/nifi-bootstrap.log/):
>>>
>>> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler] org.apache.nifi.StdErr Error: *To use 'FlightRecorder', first unlock using -XX:+UnlockCommercialFeatures.*
>>> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler] org.apache.nifi.StdErr Error: Could not create the Java Virtual Machine.
>>> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler] org.apache.nifi.StdErr Error: A fatal exception has occurred. Program will exit.
>>> 2016-12-14 10:39:36,507 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi never started. Will not restart NiFi
>>>
>>> I tried using all options as one (in case the order is disturbed, which it was):
>>>
>>> java.arg.15=-XX:+UnlockCommercialFeatures-XX:+FlightRecorder-XX:StartFlightRecording=duration=120m,filename=recording.jfr
>>>
>>>
>>> and then I get:
>>>
>>> *2016-12-14 10:50:07,574 ERROR [NiFi logging handler] org.apache.nifi.StdErr Unrecognized VM option 'UnlockCommercialFeatures -XX:+FlightRecorder -XX:StartFlightRecording=duration=120m,filename=recording.jfr'*
>>> 2016-12-14 10:50:07,574 ERROR [NiFi logging handler] org.apache.nifi.StdErr Error: Could not create the Java Virtual Machine.
>>> 2016-12-14 10:50:07,574 ERROR [NiFi logging handler] org.apache.nifi.StdErr Error: A fatal exception has occurred. Program will exit.
>>> 2016-12-14 10:50:07,598 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi never started. Will not restart NiFi
>>>
>>> Here's the second command line from /logs/nifi-bootstrap.log/, which I've wrapped for convenience in reading (see JVM version here too):
>>>
>>> 2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command Working Directory: /home/russ/dev/nifi/nifi-1.1.0
>>> 2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command Command: */home/russ/dev/jdk1.8.0_112/bin/java*
>>> -classpath /home/russ/dev/nifi/nifi-1.1.0/./conf:/home/russ/dev/nifi/nifi-1.1.0/./lib/jcl-over-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/logback-core-1.1.3.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-documentation-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-nar-utils-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/logback-classic-1.1.3.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/jul-to-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-runtime-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/slf4j-api-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-api-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-framework-api-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/log4j-over-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-properties-1.1.0.jar
>>> -Dorg.apache.jasper.compiler.disablejsr199=true
>>> -Xmx512m
>>> -Xms512m
>>> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=9191
>>> *-XX:+UnlockCommercialFeatures
>>> -XX:+FlightRecorder
>>> -XX:StartFlightRecording=duration=120m,filename=recording.jfr*
>>> -Dsun.net.http.allowRestrictedHeaders=true
>>> -Djava.net.preferIPv4Stack=true
>>> -Djava.awt.headless=true
>>> -XX:+UseG1GC
>>> -Djava.protocol.handler.pkgs=sun.net.www.protocol
>>> -Dnifi.properties.file.path=/home/russ/dev/nifi/nifi-1.1.0/./conf/nifi.properties
>>> -Dnifi.bootstrap.listen.port=41736
>>> -Dapp=NiFi
>>> -Dorg.apache.nifi.bootstrap.config.log.dir=/home/russ/dev/nifi/nifi-1.1.0/logs org.apache.nifi.NiFi
>>>
>>> Last, I frantically tried this:
>>>
>>> java.arg.15="-XX:+UnlockCommercialFeatures -XX:+FlightRecorder
>>> -XX:StartFlightRecording=duration=120m,filename=recording.jfr"
>>>
>>>
>>> and then got:
>>>
>>> *2016-12-14 11:01:36,659 ERROR [NiFi logging handler] org.apache.nifi.StdErr Error: Could not find or load main class "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder -XX:StartFlightRecording=duration=120m,filename=recording.jfr"*
>>>
>>>
>>> What's the canonical way of adding/inserting my own JVM commands? Alternatively, is NiFi analyzing and blocking -XX:arguments to the JVM as illegal or unrecognized?
>>>
>>> Thanks,
>>>
>>> Russ
Re: Adding JVM arguments when running NiFi...
Posted by Mark Payne <ma...@hotmail.com>.
Russ,
Sorry - I didn't see this sooner. That is accurate, that 'N' can be non-numeric. By default, in bootstrap.conf, there is actually a property named "java.arg.debug=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000" but it is commented out.
I think that if we want to guarantee ordering, we need to do some sort of sorting on the keys. For example, first comes all numbers in numeric ordering (i.e., - not lexigraphical ordering - so java.arg.1, java.arg.2, java.arg.3, ..., java.arg.10, java.arg.11, ...), followed by non-numeric values in lexigraphical ordering. This would make it much easier to guarantee ordering by simply using numbers but doesn't enforce numbers to be used. We didn't enforce numbers to be used because historically we've found that people are inclined to always use a one-up number. So when they want to add a new property, they go searching through the file looking for the latest number used and then use the next number; then, if they un-comment a line, that one may conflict resulting in two arguments with the same numbers and one of them is overridden.
Thanks
-Mark
> On Jan 18, 2017, at 10:22 AM, Russell Bateman <ru...@windofkeltia.com> wrote:
>
> For my own purposes, I fixed this in 0.7.1 /nifi-bootstrap///src/main/java/org/////apache/nifi/bootstrap/RunNiFi.java/, but at Joe's suggestion, I created JIRA NIFI-3364 hoping some committer will get to it for all code branches-forward. The solution I used was made to be as little disruptive to the existing code as possible. To that JIRA I attached the solution pinched between comments in a larger test demonstrating it.
>
> I had found that I could trick the order in my local environment by trial and error, however, moving to another host and with the additional need to dominate the order of JMX remote definitions as well (in order to reach NiFi remote from Java Mission Control), it just became impossible to achieve it.
>
> The code I wrote isn't the only way to fix nor even necessarily the best, but it works supporting both the ordering of /N/in java.arg.Nas well as (did you know this?) the use of non-numeric options for the position of /N/.
>
> Russ
>
> On 12/14/2016 11:05 AM, Russell Bateman wrote:
>>
>> I've doctored /conf/bootstrap.conf/ to contain these additional lines:
>>
>> java.arg.15=-XX:+UnlockCommercialFeatures
>> java.arg.16=-XX:+FlightRecorder
>> java.arg.17=-XX:StartFlightRecording=duration=120m,filename=recording.jfr
>>
>>
>> In the end, NiFi's grumpy about this and won't start (from /logs/nifi-bootstrap.log/):
>>
>> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler] org.apache.nifi.StdErr Error: *To use 'FlightRecorder', first unlock using -XX:+UnlockCommercialFeatures.*
>> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler] org.apache.nifi.StdErr Error: Could not create the Java Virtual Machine.
>> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler] org.apache.nifi.StdErr Error: A fatal exception has occurred. Program will exit.
>> 2016-12-14 10:39:36,507 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi never started. Will not restart NiFi
>>
>> I tried using all options as one (in case the order is disturbed, which it was):
>>
>> java.arg.15=-XX:+UnlockCommercialFeatures-XX:+FlightRecorder-XX:StartFlightRecording=duration=120m,filename=recording.jfr
>>
>>
>> and then I get:
>>
>> *2016-12-14 10:50:07,574 ERROR [NiFi logging handler] org.apache.nifi.StdErr Unrecognized VM option 'UnlockCommercialFeatures -XX:+FlightRecorder -XX:StartFlightRecording=duration=120m,filename=recording.jfr'*
>> 2016-12-14 10:50:07,574 ERROR [NiFi logging handler] org.apache.nifi.StdErr Error: Could not create the Java Virtual Machine.
>> 2016-12-14 10:50:07,574 ERROR [NiFi logging handler] org.apache.nifi.StdErr Error: A fatal exception has occurred. Program will exit.
>> 2016-12-14 10:50:07,598 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi never started. Will not restart NiFi
>>
>> Here's the second command line from /logs/nifi-bootstrap.log/, which I've wrapped for convenience in reading (see JVM version here too):
>>
>> 2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command Working Directory: /home/russ/dev/nifi/nifi-1.1.0
>> 2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command Command: */home/russ/dev/jdk1.8.0_112/bin/java*
>> -classpath /home/russ/dev/nifi/nifi-1.1.0/./conf:/home/russ/dev/nifi/nifi-1.1.0/./lib/jcl-over-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/logback-core-1.1.3.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-documentation-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-nar-utils-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/logback-classic-1.1.3.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/jul-to-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-runtime-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/slf4j-api-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-api-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-framework-api-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/log4j-over-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-properties-1.1.0.jar
>> -Dorg.apache.jasper.compiler.disablejsr199=true
>> -Xmx512m
>> -Xms512m
>> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=9191
>> *-XX:+UnlockCommercialFeatures
>> -XX:+FlightRecorder
>> -XX:StartFlightRecording=duration=120m,filename=recording.jfr*
>> -Dsun.net.http.allowRestrictedHeaders=true
>> -Djava.net.preferIPv4Stack=true
>> -Djava.awt.headless=true
>> -XX:+UseG1GC
>> -Djava.protocol.handler.pkgs=sun.net.www.protocol
>> -Dnifi.properties.file.path=/home/russ/dev/nifi/nifi-1.1.0/./conf/nifi.properties
>> -Dnifi.bootstrap.listen.port=41736
>> -Dapp=NiFi
>> -Dorg.apache.nifi.bootstrap.config.log.dir=/home/russ/dev/nifi/nifi-1.1.0/logs org.apache.nifi.NiFi
>>
>> Last, I frantically tried this:
>>
>> java.arg.15="-XX:+UnlockCommercialFeatures -XX:+FlightRecorder
>> -XX:StartFlightRecording=duration=120m,filename=recording.jfr"
>>
>>
>> and then got:
>>
>> *2016-12-14 11:01:36,659 ERROR [NiFi logging handler] org.apache.nifi.StdErr Error: Could not find or load main class "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder -XX:StartFlightRecording=duration=120m,filename=recording.jfr"*
>>
>>
>> What's the canonical way of adding/inserting my own JVM commands? Alternatively, is NiFi analyzing and blocking -XX:arguments to the JVM as illegal or unrecognized?
>>
>> Thanks,
>>
>> Russ
>
Re: Adding JVM arguments when running NiFi...
Posted by Russell Bateman <ru...@windofkeltia.com>.
For my own purposes, I fixed this in 0.7.1
/nifi-bootstrap///src/main/java/org/////apache/nifi/bootstrap/RunNiFi.java/,
but at Joe's suggestion, I created JIRA NIFI-3364 hoping some committer
will get to it for all code branches-forward. The solution I used was
made to be as little disruptive to the existing code as possible. To
that JIRA I attached the solution pinched between comments in a larger
test demonstrating it.
I had found that I could trick the order in my local environment by
trial and error, however, moving to another host and with the additional
need to dominate the order of JMX remote definitions as well (in order
to reach NiFi remote from Java Mission Control), it just became
impossible to achieve it.
The code I wrote isn't the only way to fix nor even necessarily the
best, but it works supporting both the ordering of /N/in java.arg.Nas
well as (did you know this?) the use of non-numeric options for the
position of /N/.
Russ
On 12/14/2016 11:05 AM, Russell Bateman wrote:
>
> I've doctored /conf/bootstrap.conf/ to contain these additional lines:
>
> java.arg.15=-XX:+UnlockCommercialFeatures
> java.arg.16=-XX:+FlightRecorder
> java.arg.17=-XX:StartFlightRecording=duration=120m,filename=recording.jfr
>
>
> In the end, NiFi's grumpy about this and won't start (from
> /logs/nifi-bootstrap.log/):
>
> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
> org.apache.nifi.StdErr Error: *To use 'FlightRecorder', first unlock
> using -XX:+UnlockCommercialFeatures.*
> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
> org.apache.nifi.StdErr Error: Could not create the Java Virtual Machine.
> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler]
> org.apache.nifi.StdErr Error: A fatal exception has occurred. Program
> will exit.
> 2016-12-14 10:39:36,507 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> NiFi never started. Will not restart NiFi
>
> I tried using all options as one (in case the order is disturbed,
> which it was):
>
> java.arg.15=-XX:+UnlockCommercialFeatures-XX:+FlightRecorder-XX:StartFlightRecording=duration=120m,filename=recording.jfr
>
>
> and then I get:
>
> *2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
> org.apache.nifi.StdErr Unrecognized VM option
> 'UnlockCommercialFeatures -XX:+FlightRecorder
> -XX:StartFlightRecording=duration=120m,filename=recording.jfr'*
> 2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
> org.apache.nifi.StdErr Error: Could not create the Java Virtual Machine.
> 2016-12-14 10:50:07,574 ERROR [NiFi logging handler]
> org.apache.nifi.StdErr Error: A fatal exception has occurred. Program
> will exit.
> 2016-12-14 10:50:07,598 INFO [main] org.apache.nifi.bootstrap.RunNiFi
> NiFi never started. Will not restart NiFi
>
> Here's the second command line from /logs/nifi-bootstrap.log/, which
> I've wrapped for convenience in reading (see JVM version here too):
>
> 2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command
> Working Directory: /home/russ/dev/nifi/nifi-1.1.0
> 2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command
> Command: */home/russ/dev/jdk1.8.0_112/bin/java*
> -classpath
> /home/russ/dev/nifi/nifi-1.1.0/./conf:/home/russ/dev/nifi/nifi-1.1.0/./lib/jcl-over-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/logback-core-1.1.3.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-documentation-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-nar-utils-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/logback-classic-1.1.3.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/jul-to-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-runtime-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/slf4j-api-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-api-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-framework-api-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/log4j-over-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-properties-1.1.0.jar
> -Dorg.apache.jasper.compiler.disablejsr199=true
> -Xmx512m
> -Xms512m
> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=9191
> *-XX:+UnlockCommercialFeatures
> -XX:+FlightRecorder
> -XX:StartFlightRecording=duration=120m,filename=recording.jfr*
> -Dsun.net.http.allowRestrictedHeaders=true
> -Djava.net.preferIPv4Stack=true
> -Djava.awt.headless=true
> -XX:+UseG1GC
> -Djava.protocol.handler.pkgs=sun.net.www.protocol
> -Dnifi.properties.file.path=/home/russ/dev/nifi/nifi-1.1.0/./conf/nifi.properties
> -Dnifi.bootstrap.listen.port=41736
> -Dapp=NiFi
> -Dorg.apache.nifi.bootstrap.config.log.dir=/home/russ/dev/nifi/nifi-1.1.0/logs
> org.apache.nifi.NiFi
>
> Last, I frantically tried this:
>
> java.arg.15="-XX:+UnlockCommercialFeatures -XX:+FlightRecorder
> -XX:StartFlightRecording=duration=120m,filename=recording.jfr"
>
>
> and then got:
>
> *2016-12-14 11:01:36,659 ERROR [NiFi logging handler]
> org.apache.nifi.StdErr Error: Could not find or load main class
> "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder
> -XX:StartFlightRecording=duration=120m,filename=recording.jfr"*
>
>
> What's the canonical way of adding/inserting my own JVM commands?
> Alternatively, is NiFi analyzing and blocking -XX:arguments to the JVM
> as illegal or unrecognized?
>
> Thanks,
>
> Russ