You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by zenfenan <gi...@git.apache.org> on 2018/01/26 04:49:24 UTC
[GitHub] nifi pull request #2436: Nifi 4819: Added support to delete blob from Azure ...
GitHub user zenfenan opened a pull request:
https://github.com/apache/nifi/pull/2436
Nifi 4819: Added support to delete blob from Azure Storage container
Implemented basic version that takes care of deleting an Azure blob from Storage container, if the blob exists. Also implemented Test.
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
- [x] Is your initial contribution a single, squashed commit?
### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zenfenan/nifi NIFI-4819
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/2436.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2436
----
commit d1290d03e68cac77e6e1f332368bbadc8ced9043
Author: zenfenaan <si...@...>
Date: 2018-01-12T17:15:13Z
NIFI-4770 ListAzureBlobStorage now properly writes azure.container flowfile attribute
commit 615f3ebc10d94f37c7463ae2d240582397e6ce8c
Author: zenfenan <si...@...>
Date: 2018-01-23T16:30:38Z
Merge branch 'master' of https://github.com/apache/nifi
commit f597a9b7aa11a48ff46931b73da216c8426542e4
Author: zenfenan <si...@...>
Date: 2018-01-26T04:42:32Z
Merge remote-tracking branch 'nifi/master'
commit 391862cd1c32e763810bfecb8d5b813472ab96e3
Author: zenfenan <si...@...>
Date: 2018-01-26T04:41:26Z
NIFI-4819: Added DeleteAzureBlobStorage that handles deletion of blob from an Azure Storage container
----
---
[GitHub] nifi pull request #2436: NIFI-4819 Added support to delete blob from Azure S...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/2436
---
[GitHub] nifi pull request #2436: NIFI-4819 Added support to delete blob from Azure S...
Posted by pvillard31 <gi...@git.apache.org>.
Github user pvillard31 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2436#discussion_r167655116
--- Diff: nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/storage/DeleteAzureBlobStorage.java ---
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.azure.storage;
+
+import com.microsoft.azure.storage.StorageException;
+import com.microsoft.azure.storage.blob.CloudBlob;
+import com.microsoft.azure.storage.blob.CloudBlobClient;
+import com.microsoft.azure.storage.blob.CloudBlobContainer;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processors.azure.AbstractAzureBlobProcessor;
+import org.apache.nifi.processors.azure.storage.utils.AzureStorageUtils;
+
+import java.net.URISyntaxException;
+import java.util.Optional;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicReference;
+
+
+@Tags({ "azure", "microsoft", "cloud", "storage", "blob" })
+@SeeAlso({ ListAzureBlobStorage.class, FetchAzureBlobStorage.class, PutAzureBlobStorage.class})
+@CapabilityDescription("Deletes the provided blob from Azure Storage")
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+public class DeleteAzureBlobStorage extends AbstractAzureBlobProcessor {
+
+ @Override
+ public void onTrigger(ProcessContext context, ProcessSession session) throws ProcessException {
+ FlowFile flowFile = session.get();
+
+ if(flowFile == null) {
+ return;
+ }
+
+ final long startNanos = System.nanoTime();
+ String containerName = context.getProperty(AzureStorageUtils.CONTAINER).evaluateAttributeExpressions(flowFile).getValue();
+ String blobPath = context.getProperty(BLOB).evaluateAttributeExpressions(flowFile).getValue();
+
+ AtomicReference<Exception> storedException = new AtomicReference<>();
--- End diff --
Is it really needed?
---
[GitHub] nifi issue #2436: NIFI-4819 Added support to delete blob from Azure Storage ...
Posted by pvillard31 <gi...@git.apache.org>.
Github user pvillard31 commented on the issue:
https://github.com/apache/nifi/pull/2436
+1, thanks for the changes @zenfenan, merging to master.
---
[GitHub] nifi issue #2436: NIFI-4819 Added support to delete blob from Azure Storage ...
Posted by zenfenan <gi...@git.apache.org>.
Github user zenfenan commented on the issue:
https://github.com/apache/nifi/pull/2436
@pvillard31 Actually I left `AtomicReference` on purpose. The reason was I wanted to implement some features later that would need that but yeah it doesn't concern or matter for the current commit/Jira so I took that off as well as fixed to `@InputRequirement(Requirement.INPUT_REQUIRED)`
---
[GitHub] nifi pull request #2436: NIFI-4819 Added support to delete blob from Azure S...
Posted by pvillard31 <gi...@git.apache.org>.
Github user pvillard31 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2436#discussion_r167655333
--- Diff: nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/storage/DeleteAzureBlobStorage.java ---
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.azure.storage;
+
+import com.microsoft.azure.storage.StorageException;
+import com.microsoft.azure.storage.blob.CloudBlob;
+import com.microsoft.azure.storage.blob.CloudBlobClient;
+import com.microsoft.azure.storage.blob.CloudBlobContainer;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processors.azure.AbstractAzureBlobProcessor;
+import org.apache.nifi.processors.azure.storage.utils.AzureStorageUtils;
+
+import java.net.URISyntaxException;
+import java.util.Optional;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicReference;
+
+
+@Tags({ "azure", "microsoft", "cloud", "storage", "blob" })
+@SeeAlso({ ListAzureBlobStorage.class, FetchAzureBlobStorage.class, PutAzureBlobStorage.class})
+@CapabilityDescription("Deletes the provided blob from Azure Storage")
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
--- End diff --
Just
````java
@InputRequirement(Requirement.INPUT_REQUIRED)
````
to be consistent.
---