You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by ssogabe <gi...@git.apache.org> on 2018/05/29 01:58:01 UTC
[GitHub] nifi pull request #2745: NIFI-5245: SimpleCSVFileLookupService should take a...
GitHub user ssogabe opened a pull request:
https://github.com/apache/nifi/pull/2745
NIFI-5245: SimpleCSVFileLookupService should take account of charset
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [ ] Has your PR been rebased against the latest commit within the target branch (typically master)?
- [ ] Is your initial contribution a single, squashed commit?
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ssogabe/nifi NiFi-5245
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/2745.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2745
----
commit 025eec69d6fa19767077b9bbef2271a66e7969ba
Author: Seiji Sogabe <s....@...>
Date: 2018-05-29T01:55:51Z
NIFI-5245: SimpleCSVFileLookupService should take account of charset
----
---
[GitHub] nifi pull request #2745: NIFI-5245: SimpleCSVFileLookupService should take a...
Posted by MikeThomsen <gi...@git.apache.org>.
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2745#discussion_r191389134
--- Diff: nifi-nar-bundles/nifi-standard-services/nifi-lookup-services-bundle/nifi-lookup-services/src/test/java/org/apache/nifi/lookup/TestCSVRecordLookupService.java ---
@@ -66,4 +66,30 @@ public void testSimpleCsvFileLookupService() throws InitializationException, IOE
assertEquals(EMPTY_RECORD, property3);
}
+ @Test
+ public void testSimpleCsvFileLookupServiceWithCharset() throws InitializationException, IOException, LookupFailureException {
+ final TestRunner runner = TestRunners.newTestRunner(TestProcessor.class);
+ final CSVRecordLookupService service = new CSVRecordLookupService();
+
+ runner.addControllerService("csv-file-lookup-service", service);
+ runner.setProperty(service, CSVRecordLookupService.CSV_FILE, "src/test/resources/test_Windows-31J.csv");
+ runner.setProperty(service, CSVRecordLookupService.CSV_FORMAT, "RFC4180");
+ runner.setProperty(service, CSVRecordLookupService.CHARSET, "Windows-31J");
+ runner.setProperty(service, CSVRecordLookupService.LOOKUP_KEY_COLUMN, "key");
+ runner.enableControllerService(service);
+ runner.assertValid(service);
+
+ final CSVRecordLookupService lookupService =
+ (CSVRecordLookupService) runner.getProcessContext()
+ .getControllerServiceLookup()
+ .getControllerService("csv-file-lookup-service");
+
+ assertThat(lookupService, instanceOf(LookupService.class));
+
+ final Optional<Record> property1 = lookupService.lookup(Collections.singletonMap("key", "property.1"));
+ assertEquals("this is property \uff11", property1.get().getAsString("value"));
--- End diff --
I would recommend adding `assertTrue(property1.isPresent())`.
---
[GitHub] nifi issue #2745: NIFI-5245: SimpleCSVFileLookupService should take account ...
Posted by MikeThomsen <gi...@git.apache.org>.
Github user MikeThomsen commented on the issue:
https://github.com/apache/nifi/pull/2745
You have two checkstyle issues that caused the build to fail. I'll start a review, but those need to be fixed.
---
[GitHub] nifi pull request #2745: NIFI-5245: SimpleCSVFileLookupService should take a...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/2745
---
[GitHub] nifi issue #2745: NIFI-5245: SimpleCSVFileLookupService should take account ...
Posted by MikeThomsen <gi...@git.apache.org>.
Github user MikeThomsen commented on the issue:
https://github.com/apache/nifi/pull/2745
While it was building here, I ran the build locally and it passed.
---
[GitHub] nifi pull request #2745: NIFI-5245: SimpleCSVFileLookupService should take a...
Posted by MikeThomsen <gi...@git.apache.org>.
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2745#discussion_r191388864
--- Diff: nifi-nar-bundles/nifi-standard-services/nifi-lookup-services-bundle/nifi-lookup-services/src/test/java/org/apache/nifi/lookup/TestCSVRecordLookupService.java ---
@@ -66,4 +66,30 @@ public void testSimpleCsvFileLookupService() throws InitializationException, IOE
assertEquals(EMPTY_RECORD, property3);
}
+ @Test
+ public void testSimpleCsvFileLookupServiceWithCharset() throws InitializationException, IOException, LookupFailureException {
+ final TestRunner runner = TestRunners.newTestRunner(TestProcessor.class);
+ final CSVRecordLookupService service = new CSVRecordLookupService();
+
+ runner.addControllerService("csv-file-lookup-service", service);
+ runner.setProperty(service, CSVRecordLookupService.CSV_FILE, "src/test/resources/test_Windows-31J.csv");
+ runner.setProperty(service, CSVRecordLookupService.CSV_FORMAT, "RFC4180");
+ runner.setProperty(service, CSVRecordLookupService.CHARSET, "Windows-31J");
+ runner.setProperty(service, CSVRecordLookupService.LOOKUP_KEY_COLUMN, "key");
+ runner.enableControllerService(service);
+ runner.assertValid(service);
+
+ final CSVRecordLookupService lookupService =
+ (CSVRecordLookupService) runner.getProcessContext()
+ .getControllerServiceLookup()
+ .getControllerService("csv-file-lookup-service");
+
+ assertThat(lookupService, instanceOf(LookupService.class));
--- End diff --
Neither is this. If you can successfully add it to the test runner and enable it, you're good to go on this.
---
[GitHub] nifi pull request #2745: NIFI-5245: SimpleCSVFileLookupService should take a...
Posted by MikeThomsen <gi...@git.apache.org>.
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2745#discussion_r191388796
--- Diff: nifi-nar-bundles/nifi-standard-services/nifi-lookup-services-bundle/nifi-lookup-services/src/test/java/org/apache/nifi/lookup/TestCSVRecordLookupService.java ---
@@ -66,4 +66,30 @@ public void testSimpleCsvFileLookupService() throws InitializationException, IOE
assertEquals(EMPTY_RECORD, property3);
}
+ @Test
+ public void testSimpleCsvFileLookupServiceWithCharset() throws InitializationException, IOException, LookupFailureException {
+ final TestRunner runner = TestRunners.newTestRunner(TestProcessor.class);
+ final CSVRecordLookupService service = new CSVRecordLookupService();
+
+ runner.addControllerService("csv-file-lookup-service", service);
+ runner.setProperty(service, CSVRecordLookupService.CSV_FILE, "src/test/resources/test_Windows-31J.csv");
+ runner.setProperty(service, CSVRecordLookupService.CSV_FORMAT, "RFC4180");
+ runner.setProperty(service, CSVRecordLookupService.CHARSET, "Windows-31J");
+ runner.setProperty(service, CSVRecordLookupService.LOOKUP_KEY_COLUMN, "key");
+ runner.enableControllerService(service);
+ runner.assertValid(service);
+
+ final CSVRecordLookupService lookupService =
--- End diff --
This isn't necessary.
---
[GitHub] nifi issue #2745: NIFI-5245: SimpleCSVFileLookupService should take account ...
Posted by ssogabe <gi...@git.apache.org>.
Github user ssogabe commented on the issue:
https://github.com/apache/nifi/pull/2745
Thanks @MikeThomsen, I'll fix it as soon as possible!
---
[GitHub] nifi pull request #2745: NIFI-5245: SimpleCSVFileLookupService should take a...
Posted by MikeThomsen <gi...@git.apache.org>.
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2745#discussion_r191388975
--- Diff: nifi-nar-bundles/nifi-standard-services/nifi-lookup-services-bundle/nifi-lookup-services/src/test/java/org/apache/nifi/lookup/TestSimpleCsvFileLookupService.java ---
@@ -64,4 +64,28 @@ public void testSimpleCsvFileLookupService() throws InitializationException, IOE
assertEquals(EMPTY_STRING, property3);
}
+ @Test
+ public void testSimpleCsvFileLookupServiceWithCharset() throws InitializationException, IOException, LookupFailureException {
+ final TestRunner runner = TestRunners.newTestRunner(TestProcessor.class);
+ final SimpleCsvFileLookupService service = new SimpleCsvFileLookupService();
+
+ runner.addControllerService("csv-file-lookup-service", service);
+ runner.setProperty(service, SimpleCsvFileLookupService.CSV_FILE, "src/test/resources/test_Windows-31J.csv");
+ runner.setProperty(service, SimpleCsvFileLookupService.CSV_FORMAT, "RFC4180");
+ runner.setProperty(service, SimpleCsvFileLookupService.CHARSET, "Windows-31J");
+ runner.setProperty(service, SimpleCsvFileLookupService.LOOKUP_KEY_COLUMN, "key");
+ runner.setProperty(service, SimpleCsvFileLookupService.LOOKUP_VALUE_COLUMN, "value");
+ runner.enableControllerService(service);
+ runner.assertValid(service);
+
+ final SimpleCsvFileLookupService lookupService =
--- End diff --
These blocks can be removed.
---
[GitHub] nifi pull request #2745: NIFI-5245: SimpleCSVFileLookupService should take a...
Posted by MikeThomsen <gi...@git.apache.org>.
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2745#discussion_r191389306
--- Diff: nifi-nar-bundles/nifi-standard-services/nifi-lookup-services-bundle/nifi-lookup-services/src/test/java/org/apache/nifi/lookup/TestSimpleCsvFileLookupService.java ---
@@ -64,4 +64,28 @@ public void testSimpleCsvFileLookupService() throws InitializationException, IOE
assertEquals(EMPTY_STRING, property3);
}
+ @Test
+ public void testSimpleCsvFileLookupServiceWithCharset() throws InitializationException, IOException, LookupFailureException {
+ final TestRunner runner = TestRunners.newTestRunner(TestProcessor.class);
+ final SimpleCsvFileLookupService service = new SimpleCsvFileLookupService();
+
+ runner.addControllerService("csv-file-lookup-service", service);
+ runner.setProperty(service, SimpleCsvFileLookupService.CSV_FILE, "src/test/resources/test_Windows-31J.csv");
+ runner.setProperty(service, SimpleCsvFileLookupService.CSV_FORMAT, "RFC4180");
+ runner.setProperty(service, SimpleCsvFileLookupService.CHARSET, "Windows-31J");
+ runner.setProperty(service, SimpleCsvFileLookupService.LOOKUP_KEY_COLUMN, "key");
+ runner.setProperty(service, SimpleCsvFileLookupService.LOOKUP_VALUE_COLUMN, "value");
+ runner.enableControllerService(service);
+ runner.assertValid(service);
+
+ final SimpleCsvFileLookupService lookupService =
+ (SimpleCsvFileLookupService) runner.getProcessContext()
+ .getControllerServiceLookup()
+ .getControllerService("csv-file-lookup-service");
+
+ assertThat(lookupService, instanceOf(LookupService.class));
+
+ final Optional<String> property1 = lookupService.lookup(Collections.singletonMap("key", "property.1"));
+ assertEquals(Optional.of("this is property \uff11"), property1);
--- End diff --
Add `assertTrue(property1.isPresent())` so the test will cleanly fail the evaluation if it's not present.
---