You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by gresockj <gi...@git.apache.org> on 2016/08/23 12:52:46 UTC
[GitHub] nifi pull request #917: NIFI-2631: Adding 'Commit Mode' and 'Use Versions' t...
GitHub user gresockj opened a pull request:
https://github.com/apache/nifi/pull/917
NIFI-2631: Adding 'Commit Mode' and 'Use Versions' to ListS3
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gresockj/nifi NIFI-2631-master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/917.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #917
----
commit 02e94ed885f86d67e02eb9b4e783ac4ef56397e4
Author: Joe Gresock <jo...@lmco.com>
Date: 2016-08-23T12:46:12Z
NIFI-2631: Adding 'Commit Mode' and 'Use Versions' to ListS3
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #917: NIFI-2631: Adding 'Commit Mode' and 'Use Versions' to ListS...
Posted by jvwing <gi...@git.apache.org>.
Github user jvwing commented on the issue:
https://github.com/apache/nifi/pull/917
@gresockj, would it be OK if we review/polish this PR first and leave the 0.x PR #916 unit we're finished? I don't want you to have to duplicate effort -- I believe the branches are fairly similar with respect to ListS3, and it may be as simple as just merging twice when we're finished. Does that match what you've experienced on the two branches so far?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #917: NIFI-2631: Adding 'Commit Mode' and 'Use Versions' to ListS...
Posted by gresockj <gi...@git.apache.org>.
Github user gresockj commented on the issue:
https://github.com/apache/nifi/pull/917
ITListS3 idea sounds good, I'll work on it when I get a chance.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #917: NIFI-2631: Adding 'Commit Mode' and 'Use Versions' to ListS...
Posted by jvwing <gi...@git.apache.org>.
Github user jvwing commented on the issue:
https://github.com/apache/nifi/pull/917
Build with contrib-check passes OK. The TravisCI failure was from memory ("GC overhead limit exceeded"), not code issues. All of my manual testing went well, both with and without versioning. The code quality looks very good.
One small thing -- the integration test ITListS3::testGetPropertyDescriptors fails due to the new property descriptors. Would you please update that test? Also, what would you think about a test or two for the versioning feature? Even if you copy/pasted one of the existing integration tests and added `runner.setProperty(ListS3.USE_VERSIONS, "true");`, running it against the non-version-enabled bucket, it would still exercise the S3VersionBucketLister code paths.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #917: NIFI-2631: Adding 'Commit Mode' and 'Use Versions' to ListS...
Posted by jvwing <gi...@git.apache.org>.
Github user jvwing commented on the issue:
https://github.com/apache/nifi/pull/917
Are you running a cluster? I have so far tested only a standalone instance.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #917: NIFI-2631: Adding 'Commit Mode' and 'Use Versions' to ListS...
Posted by gresockj <gi...@git.apache.org>.
Github user gresockj commented on the issue:
https://github.com/apache/nifi/pull/917
Thanks for checking that out, gives me some more confidence that it's unrelated to this. For now I'll proceed with the ITListS3 tests when I get a chance.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #917: NIFI-2631: Adding 'Commit Mode' and 'Use Versions' to ListS...
Posted by gresockj <gi...@git.apache.org>.
Github user gresockj commented on the issue:
https://github.com/apache/nifi/pull/917
Sounds good, I believe ListS3 is the same on both branches, so finishing this PR first should work.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #917: NIFI-2631: Adding 'Commit Mode' and 'Use Versions' to ListS...
Posted by gresockj <gi...@git.apache.org>.
Github user gresockj commented on the issue:
https://github.com/apache/nifi/pull/917
Yes, it's requesting this:
https://localhost:8443/nifi-api/controller/process-groups/66ef7ae5-4bdb-4c50-9811-429be759f6fb/processors/7738f82e-d185-3931-a98e-c1156cd21cbc/state
totalEntryCount is 0 for both clusterState and localState
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #917: NIFI-2631: Adding 'Commit Mode' and 'Use Versions' t...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/917
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #917: NIFI-2631: Adding 'Commit Mode' and 'Use Versions' to ListS...
Posted by jvwing <gi...@git.apache.org>.
Github user jvwing commented on the issue:
https://github.com/apache/nifi/pull/917
@gresockj No, I have not noticed that, and I've been viewing and clearing the state along with my testing. Are there javascript errors in your browser? Are you familiar with browser debugging tools? When you open the state dialog, there should be an XHR call to the server to get this state information, something like:
http://localhost:8080/nifi-api/processors/dc5e91e6-0156-1000-ef9e-9d07ae89516c/state
And this returns JSON with the state in it. Can you check if that query is being run, and if it is failing or returning nothing?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #917: NIFI-2631: Adding 'Commit Mode' and 'Use Versions' to ListS...
Posted by jvwing <gi...@git.apache.org>.
Github user jvwing commented on the issue:
https://github.com/apache/nifi/pull/917
I tried a simple cluster running in a docker container, and state seemed to work just fine. I tested ListS3 running both on the primary node only and across the cluster nodes. The state API json reflected clustered state in both cases.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #917: NIFI-2631: Adding 'Commit Mode' and 'Use Versions' to ListS...
Posted by jvwing <gi...@git.apache.org>.
Github user jvwing commented on the issue:
https://github.com/apache/nifi/pull/917
Thanks for the test upgrades, @gresockj, they look good. I will merge shortly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #917: NIFI-2631: Adding 'Commit Mode' and 'Use Versions' to ListS...
Posted by gresockj <gi...@git.apache.org>.
Github user gresockj commented on the issue:
https://github.com/apache/nifi/pull/917
Yeah, I'm on a cluster. I'll keep digging.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #917: NIFI-2631: Adding 'Commit Mode' and 'Use Versions' to ListS...
Posted by gresockj <gi...@git.apache.org>.
Github user gresockj commented on the issue:
https://github.com/apache/nifi/pull/917
Hmm, so this is strange. I was testing ListS3 out in NiFi today and I noticed that I cannot see the state in the View State window. However, the processor behaves as if the state is being persisted (i.e., it won't list any flow files until I actually put another object in the bucket). However, other processors allow me to see the state in the View State window. I'm positive I was able to see this state when I initially tested the processor in NiFi, but I can't think of what might have changed. Have you experienced this at all, @jvwing?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---