You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by GitBox <gi...@apache.org> on 2022/05/20 12:38:04 UTC
[GitHub] [nifi] markobean commented on a diff in pull request #6061: NIFI-5378 prevent duplicate keys with different values in nifi.p…
markobean commented on code in PR #6061:
URL: https://github.com/apache/nifi/pull/6061#discussion_r878099667
##########
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-properties-loader/src/main/java/org/apache/nifi/properties/NiFiPropertiesLoader.java:
##########
@@ -211,6 +220,45 @@ public NiFiProperties get() {
return instance;
}
+ private void checkForDuplicates(File file) {
+ Map<String, String> properties = new HashMap<>();
+ Map<String, Set<String>> duplicateProperties = new HashMap<>();
+ Pattern keyPattern = Pattern.compile("([^!#=][^=]*)=(.*)");
+
+ if (file == null) {
+ throw new IllegalArgumentException("NiFi properties file missing or unreadable");
+ }
+ // Scan the properties file for duplicate keys
+ try (BufferedReader bufferedReader = new BufferedReader(new FileReader(file))) {
+ String line;
+ while ((line = bufferedReader.readLine()) != null) {
+ Matcher matcher = keyPattern.matcher(line);
+
+ if (matcher.matches()) {
+ String key = matcher.group(1);
+ String value = matcher.group(2);
+
+ String existingValue = properties.put(key, value);
+
+ if (existingValue != null && !existingValue.equals(value)) {
+ Set<String> dupes = duplicateProperties.computeIfAbsent(key, k -> new HashSet<>(Collections.singleton(existingValue)));
+ dupes.add(value);
+ }
+ }
+ }
+ } catch (IOException ioe) {
+ throw new IllegalArgumentException("Cannot load properties file [" + file.getAbsolutePath() + "] due to "
+ + ioe.getLocalizedMessage(), ioe);
+ }
+
+ if (!duplicateProperties.isEmpty()) {
+ duplicateProperties.keySet().forEach(
+ k -> logger.error("Duplicate values found for key '{}': {}", k, duplicateProperties.get(k))
+ );
+ throw new IllegalArgumentException("Duplicate keys were detected in the properties file. See previous errors.");
Review Comment:
This is not really a concern. NiFi property loading is just about the first thing to happen. None of the other chatty log messages are entered. The following is an example of the entire nifi-app.log when a duplicate property is found in nifi.properties - and specifically this example duplicated the last property in the file:
2022-05-20 08:34:43,428 INFO [main] org.apache.nifi.NiFi Launching NiFi...
2022-05-20 08:34:43,465 ERROR [main] o.a.nifi.properties.NiFiPropertiesLoader Duplicate values found for key 'nifi.diagnostics.on.shutdown.max.directory.size': [20 MB, 10 MB]
2022-05-20 08:34:43,469 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi
java.lang.IllegalArgumentException: There was an issue decrypting protected properties
at org.apache.nifi.NiFi.initializeProperties(NiFi.java:373)
at org.apache.nifi.NiFi.convertArgumentsToValidatedNiFiProperties(NiFi.java:341)
at org.apache.nifi.NiFi.convertArgumentsToValidatedNiFiProperties(NiFi.java:337)
at org.apache.nifi.NiFi.main(NiFi.java:329)
Caused by: java.lang.IllegalArgumentException: Duplicate keys were detected in the properties file. See previous errors.
at org.apache.nifi.properties.NiFiPropertiesLoader.checkForDuplicates(NiFiPropertiesLoader.java:258)
at org.apache.nifi.properties.NiFiPropertiesLoader.load(NiFiPropertiesLoader.java:159)
at org.apache.nifi.properties.NiFiPropertiesLoader.load(NiFiPropertiesLoader.java:199)
at org.apache.nifi.properties.NiFiPropertiesLoader.loadDefault(NiFiPropertiesLoader.java:263)
at org.apache.nifi.properties.NiFiPropertiesLoader.getDefaultProperties(NiFiPropertiesLoader.java:267)
at org.apache.nifi.properties.NiFiPropertiesLoader.get(NiFiPropertiesLoader.java:217)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.nifi.NiFi.initializeProperties(NiFi.java:368)
... 3 common frames omitted
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org