You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2021/02/05 11:45:14 UTC
[GitHub] [hadoop] mukund-thakur commented on a change in pull request #2667: HADOOP-16906. Add Abortable.abort() interface for streams to enable output stream to be terminated
mukund-thakur commented on a change in pull request #2667:
URL: https://github.com/apache/hadoop/pull/2667#discussion_r570770903
##########
File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ABlockOutputStream.java
##########
@@ -541,6 +542,10 @@ public boolean hasCapability(String capability) {
case StreamCapabilities.IOSTATISTICS:
return true;
+ // S3A supports abort.
+ case StreamCapabilities.ABORTABLE:
Review comment:
We can merge both case statements
##########
File path: hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ABlockOutputArray.java
##########
@@ -155,4 +156,22 @@ public void testMarkReset() throws Throwable {
markAndResetDatablock(createFactory(getFileSystem()));
}
+ @Test
+ public void testAbortAfterWrite() throws Throwable {
+ Path dest = path("testAbortAfterWrite");
Review comment:
use getMethodName()
##########
File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ABlockOutputStream.java
##########
@@ -551,6 +556,24 @@ public IOStatistics getIOStatistics() {
return iostatistics;
}
+ @Override
+ public void abort() {
+ if (closed.getAndSet(true)) {
+ // already closed
+ LOG.debug("Ignoring abort() as stream is already closed");
+ return;
+ }
+
+ S3ADataBlocks.DataBlock block = getActiveBlock();
+ try {
+ if (multiPartUpload != null) {
Review comment:
Got it,we are closing the stream before only.
##########
File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ABlockOutputStream.java
##########
@@ -551,6 +556,24 @@ public IOStatistics getIOStatistics() {
return iostatistics;
}
+ @Override
+ public void abort() {
+ if (closed.getAndSet(true)) {
+ // already closed
+ LOG.debug("Ignoring abort() as stream is already closed");
+ return;
+ }
+
+ S3ADataBlocks.DataBlock block = getActiveBlock();
+ try {
+ if (multiPartUpload != null) {
Review comment:
Wondering what happens in case of non multipart upload
##########
File path: hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ABlockOutputArray.java
##########
@@ -155,4 +156,22 @@ public void testMarkReset() throws Throwable {
markAndResetDatablock(createFactory(getFileSystem()));
}
+ @Test
+ public void testAbortAfterWrite() throws Throwable {
+ Path dest = path("testAbortAfterWrite");
+ describe(" testAbortAfterWrite");
Review comment:
nit: A bit more explanatory.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org