[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6644: NIFI-10789: set flowfile attributes upon failure/error when fetching …

[GitBox <gi...@apache.org>]

exceptionfactory commented on code in PR #6644:
URL: https://github.com/apache/nifi/pull/6644#discussion_r1019678331


##########
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/storage/FetchAzureDataLakeStorage.java:
##########
@@ -124,14 +134,25 @@ public void onTrigger(ProcessContext context, ProcessSession session) throws Pro
                 throw new ProcessException(FILE.getDisplayName() + " (" + fileName + ") points to a directory. Full path: " + fileClient.getFilePath());
             }
 
+
             flowFile = session.write(flowFile, os -> fileClient.readWithResponse(os, fileRange, retryOptions, null, false, null, Context.NONE));
             session.getProvenanceReporter().modifyContent(flowFile);
             session.transfer(flowFile, REL_SUCCESS);
 
             final long transferMillis = TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - startNanos);
             session.getProvenanceReporter().fetch(flowFile, fileClient.getFileUrl(), transferMillis);
+        } catch (final DataLakeStorageException e) {
+            getLogger().error("Failure to fetch file from Azure Data Lake Storage", e);
+            flowFile = session.putAttribute(flowFile, "azure.datalake.storage.statusCode", String.valueOf(e.getStatusCode()));
+            flowFile = session.putAttribute(flowFile, "azure.datalake.storage.errorCode", e.getErrorCode());
+            flowFile = session.putAttribute(flowFile, "azure.datalake.storage.serviceMessage", e.getServiceMessage());
+            flowFile = session.putAttribute(flowFile, "azure.datalake.storage.errorMessage", e.getMessage());
+            flowFile = session.penalize(flowFile);
+            session.transfer(flowFile, REL_FAILURE);
         } catch (Exception e) {
             getLogger().error("Failure to fetch file from Azure Data Lake Storage", e);
+            // other exception, set a generic failure reason flowfile attribute
+            flowFile = session.putAttribute(flowFile, "azure.datalake.storage.failure.reason", e.getMessage());

Review Comment:
   Recommend naming this `azure.datalake.storage.errorMessage` to match the other exception attribute.
   ```suggestion
               flowFile = session.putAttribute(flowFile, "azure.datalake.storage.errorMessage", e.getMessage());
   ```



##########
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/storage/FetchAzureDataLakeStorage.java:
##########
@@ -48,6 +51,13 @@
 @SeeAlso({PutAzureDataLakeStorage.class, DeleteAzureDataLakeStorage.class, ListAzureDataLakeStorage.class})
 @CapabilityDescription("Fetch the provided file from Azure Data Lake Storage")
 @InputRequirement(Requirement.INPUT_REQUIRED)
+@WritesAttributes({
+        @WritesAttribute(attribute = "azure.datalake.storage.statusCode", description = "The HTTP error code (if available) from the failed operation"),
+        @WritesAttribute(attribute = "azure.datalake.storage.errorCode", description = "The Azure Data Lake Storage moniker of the failed operation"),
+        @WritesAttribute(attribute = "azure.datalake.storage.serviceMessage", description = "The Azure Data Lake Storage service message from the failed operation"),
+        @WritesAttribute(attribute = "azure.datalake.storage.errorMessage", description = "The Azure Data Lake Storage error message from the failed operation"),
+        @WritesAttribute(attribute = "azure.datalake.storage.failure.reason", description = "An error message to indicate the cause of the failed operation")

Review Comment:
   Recommend removing this attribute and consolidating on the usage of `azure.datalake.storage.errorMessage`:
   ```suggestion
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org