You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by st...@apache.org on 2020/09/09 11:02:02 UTC
[hadoop] branch branch-3.3 updated: HADOOP-17181. Handle transient
stream read failures in FileSystem contract tests (#2286)
This is an automated email from the ASF dual-hosted git repository.
stevel pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/hadoop.git
The following commit(s) were added to refs/heads/branch-3.3 by this push:
new 262c575 HADOOP-17181. Handle transient stream read failures in FileSystem contract tests (#2286)
262c575 is described below
commit 262c575fab9a032d38c4d9329ae30368590915b9
Author: Steve Loughran <st...@cloudera.com>
AuthorDate: Wed Sep 9 11:58:26 2020 +0100
HADOOP-17181. Handle transient stream read failures in FileSystem contract tests (#2286)
Contributed by Steve Loughran.
* Fixes AbstractContractSeekTest test to use readFully
* Doesn't do this to AbstractContractUnbufferTest test as it changes the test too much.
Instead just notes in the error that this may be transient
The issue is that read(buffer) doesn't guarantee that the buffer is filled, only that it will
read up to a point, and that may be just be the amount of data left in the TCP packet.
readFully corrects for this, but using it in the unbuffer test runs the risk that what
is tested for in terms of unbuffering doesn't actually get validated.
Change-Id: I046eadb69b80ba0aac468b354c82c4d510dc3699
---
.../java/org/apache/hadoop/fs/contract/AbstractContractSeekTest.java | 5 ++---
.../org/apache/hadoop/fs/contract/AbstractContractUnbufferTest.java | 3 ++-
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractSeekTest.java b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractSeekTest.java
index ca8e4a0..d341784 100644
--- a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractSeekTest.java
+++ b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractSeekTest.java
@@ -317,7 +317,7 @@ public abstract class AbstractContractSeekTest extends AbstractFSContractTestBas
int v = 256;
byte[] readBuffer = new byte[v];
- assertEquals(v, instream.read(128, readBuffer, 0, v));
+ instream.readFully(128, readBuffer, 0, v);
//have gone back
assertEquals(40000, instream.getPos());
//content is the same too
@@ -572,8 +572,7 @@ public abstract class AbstractContractSeekTest extends AbstractFSContractTestBas
// now read the entire file in one go
byte[] fullFile = new byte[TEST_FILE_LEN];
- assertEquals(TEST_FILE_LEN,
- instream.read(0, fullFile, 0, fullFile.length));
+ instream.readFully(0, fullFile, 0, fullFile.length);
assertEquals(0, instream.getPos());
// now read past the end of the file
diff --git a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractUnbufferTest.java b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractUnbufferTest.java
index 5eb1e89..adaf0a9 100644
--- a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractUnbufferTest.java
+++ b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractUnbufferTest.java
@@ -137,7 +137,8 @@ public abstract class AbstractContractUnbufferTest extends AbstractFSContractTes
throws IOException {
byte[] streamData = new byte[length];
assertEquals("failed to read expected number of bytes from "
- + "stream", length, stream.read(streamData));
+ + "stream. This may be transient",
+ length, stream.read(streamData));
byte[] validateFileBytes;
if (startIndex == 0 && length == fileBytes.length) {
validateFileBytes = fileBytes;
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org