You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by mgaido91 <gi...@git.apache.org> on 2018/03/19 14:43:56 UTC
[GitHub] nifi pull request #2565: NIFI-4631: Use java.nio.file.Files in ListFile to i...
GitHub user mgaido91 opened a pull request:
https://github.com/apache/nifi/pull/2565
NIFI-4631: Use java.nio.file.Files in ListFile to improve performance
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
- [x] Is your initial contribution a single, squashed commit?
### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes? Performed a perf test on a random generated tree of about 100.000 nested directories: with the new code it takes ~2.5s against ~8s of the old code.
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? NA
- [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? NA
- [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? NA
- [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? NA
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered? NA
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
For the reason of the choice of the implementation, please refer to https://github.com/brettryan/io-recurse-tests and https://stackoverflow.com/questions/2056221/recursively-list-files-in-java.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mgaido91/nifi NIFI-4631
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/2565.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2565
----
commit fc039b7738a65a4559d68da768ba3b38077a61ea
Author: Marco Gaido <ma...@...>
Date: 2018-03-19T13:48:27Z
NIFI-4631: Use java.nio.file.Files in ListFile to improve performance
----
---
[GitHub] nifi pull request #2565: NIFI-4631: Use java.nio.file.Files in ListFile to i...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/2565
---
[GitHub] nifi issue #2565: NIFI-4631: Use java.nio.file.Files in ListFile to improve ...
Posted by mgaido91 <gi...@git.apache.org>.
Github user mgaido91 commented on the issue:
https://github.com/apache/nifi/pull/2565
@markap14 I don't know the reason but it might be because I run it on a deeply nested structure made basically all of folders. Probably the new solution is particularly more efficient in that case.
---
[GitHub] nifi issue #2565: NIFI-4631: Use java.nio.file.Files in ListFile to improve ...
Posted by markap14 <gi...@git.apache.org>.
Github user markap14 commented on the issue:
https://github.com/apache/nifi/pull/2565
@mgaido91 thanks for the update! I ran some tests locally on my laptop as well. I didn't see as big a difference as you mentioned in the JIRA, but I did see a performance improvement, after running several times. The performance improvement was more pronounced, it seems, when there was a large nested structure, which I suppose makes sense. I've merged this to master. Thanks again for the update!
---