[GitHub] [nifi] timeabarna commented on a change in pull request #5098: NIFI-8248 Modify PutAzureDataLakeStorage processor to use temp file

[GitBox <gi...@apache.org>]

timeabarna commented on a change in pull request #5098:
URL: https://github.com/apache/nifi/pull/5098#discussion_r655949768



##########
File path: nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/storage/ListAzureDataLakeStorage.java
##########
@@ -129,6 +131,16 @@
             .expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
             .build();
 
+    public static final PropertyDescriptor TEMP_FILE_PREFIX = new PropertyDescriptor.Builder()
+            .name("azure-temp-file-prefix")
+            .displayName("Temp File Prefix for Azure")

Review comment:
       @exceptionfactory and @jfrazee Thank you very much for your help. I've implemented the requested changes however during the unit tests it has brought up another perspective I would like to share. 
   Combining the file filter with the temp prefix would mean we would like to have a regex which would like to include only the specified files and exclude some specified files, I think this is very confusing for the user and may not be a best practice.
   As we need to provide the option for prefix override it means that sometimes separate Regex expressions must be concatenated. I have no knowledge regarding our Users' Regex experience level however combining two separate, conflicting scoped (for example a negative lookahead and a wildcard word match) Regex Expression can be challenging even for Developers.
   My last concern is what if the Processor already has a path filter set. It would override the new default filter and temp files started appear in the listing which may impact the existing flows.
   So for the best User experience I would like to recommend to keep the prefix property which serves as the default value and the override possibility however change it to accept a Regex expression. This way our Users have the control to change only the filter they have to in a straightforward and convenient way.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org