You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by alopresto <gi...@git.apache.org> on 2017/03/09 01:37:38 UTC
[GitHub] nifi pull request #1579: NIFI-3313 Added explicit Java runtime argument to d...
GitHub user alopresto opened a pull request:
https://github.com/apache/nifi/pull/1579
NIFI-3313 Added explicit Java runtime argument to default bootstrap.c\u2026
\u2026onf to avoid blocking on VM deployment.
This PR needs review in a specific environment. The reported issue is that NiFi running in a container or Virtual Machine environment that does not have access to sufficient entropy will block indeterminately on startup, right after the "Loaded *n* properties" message:
```
2017-03-08 16:38:07,479 INFO [main] org.apache.nifi.NiFi Launching NiFi...
2017-03-08 16:38:07,656 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Determined default nifi.properties path to be '/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.2.0-SNAPSHOT-bin/nifi-1.2.0-SNAPSHOT/./conf/nifi.properties'
2017-03-08 16:38:07,659 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Loaded 124 properties from /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.2.0-SNAPSHOT-bin/nifi-1.2.0-SNAPSHOT/./conf/nifi.properties
2017-03-08 16:38:07,665 INFO [main] org.apache.nifi.NiFi Loaded 124 properties
```
I have added a Java runtime argument to `conf/bootstrap.conf` which directs Java to point the Entropy Generating Device (`java.security.egd`) to `/dev/urandom`. This is *not* a security concern because NiFi is *not* generating long-lived secrets at startup (many additional explanatory resources in NIFI-3313).
However, I cannot reproduce the original issue locally. I have tried running the application on my native OS (Mac OS X 10.11.6), in a Docker container (`aldrin/apache-nifi`) on the Boot2Docker ISO, and in a Docker container (`aldrin/apache-nifi`) on a new Ubuntu Xerial 16.04.2 LTS installation inside VirtualBox. In none of these environments could I successfully block NiFi from starting.
I request that whoever reviews this is someone who has encountered the blocking issue and can consistently reproduce it in order to ensure this change solves the problem. I have run the patched version on native OS (i.e. direct access to PRNG) and there were no ill effects.
<hr>
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [ ] Has your PR been rebased against the latest commit within the target branch (typically master)?
- [ ] Is your initial contribution a single, squashed commit?
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/alopresto/nifi NIFI-3313
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/1579.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1579
----
commit 654d616407cc7271d818b8902a17e9dafcafbb2f
Author: Andy LoPresto <al...@apache.org>
Date: 2017-03-09T00:44:49Z
NIFI-3313 Added explicit Java runtime argument to default bootstrap.conf to avoid blocking on VM deployment.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1579: NIFI-3313 Added explicit Java runtime argument to default ...
Posted by ijokarumawak <gi...@git.apache.org>.
Github user ijokarumawak commented on the issue:
https://github.com/apache/nifi/pull/1579
@alopresto I have an environment that can reproduce insufficient entropy blocking NiFi to startup. Will try to review..
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1579: NIFI-3313 Added explicit Java runtime argument to default ...
Posted by alopresto <gi...@git.apache.org>.
Github user alopresto commented on the issue:
https://github.com/apache/nifi/pull/1579
Awesome @ijokarumawak . Thanks for the extensive testing. I'll merge this and hopefully it will help people who have been experiencing long delays in startup but didn't know why.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1579: NIFI-3313 Added explicit Java runtime argument to default ...
Posted by ijokarumawak <gi...@git.apache.org>.
Github user ijokarumawak commented on the issue:
https://github.com/apache/nifi/pull/1579
@alopresto LGTM +1. Thanks for fixing this!
Please merge if my test result is sufficient.
## Confirmed that I can reproduce the issue
I had been using a workaround by mapping hosts `/dev/urandom` to Docker containers `/dev/random` before. To confirm the issue happens, I removed that workaround and started 7 NiFi containers using docker-compose.
Without the workaround, NiFi start-up was blocked exactly the same point as expected, and here is the stacktrace of main method:
```
Without workaround main thread blocks here:
"main" Id=1 RUNNABLE (in native code)
at java.io.FileInputStream.readBytes(Native Method)
at java.io.FileInputStream.read(FileInputStream.java:255)
at sun.security.provider.SeedGenerator$URLSeedGenerator.getSeedBytes(SeedGenerator.java:539)
at sun.security.provider.SeedGenerator.generateSeed(SeedGenerator.java:144)
at sun.security.provider.SecureRandom$SeederHolder.<clinit>(SecureRandom.java:203)
at sun.security.provider.SecureRandom.engineNextBytes(SecureRandom.java:221)
- waiting on sun.security.provider.SecureRandom@39c08111
at java.security.SecureRandom.nextBytes(SecureRandom.java:468)
- waiting on java.security.SecureRandom@5cc41bea
at org.jasypt.salt.RandomSaltGenerator.generateSalt(RandomSaltGenerator.java:92)
- waiting on java.security.SecureRandom@5cc41bea
at org.jasypt.encryption.pbe.StandardPBEByteEncryptor.encrypt(StandardPBEByteEncryptor.java:891)
at org.jasypt.encryption.pbe.StandardPBEStringEncryptor.encrypt(StandardPBEStringEncryptor.java:642)
at org.apache.nifi.encrypt.StringEncryptor.encrypt(StringEncryptor.java:130)
at org.apache.nifi.encrypt.StringEncryptor.createEncryptor(StringEncryptor.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
```
It took about 10 minutes to start-up. Usually it takes 10 - 15 min without workaround in my environment.
## Confirmed that this PR addresses the issue
After reproducing the issue, I replaced bootstrap.conf in my NiFi container image. Then started 7 NiFi containers using docker-compose. Did the same thing 3 times. The blocking issue didn't happen, and NiFi can startup in about 1-2 minutes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1579: NIFI-3313 Added explicit Java runtime argument to d...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/1579
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---