You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2022/04/21 00:32:27 UTC
[GitHub] [hadoop] xinglin opened a new pull request, #4208: HADOOP-16768. SnappyCompressor test cases wrongly assume that the compressed data is always smaller than the input data.
xinglin opened a new pull request, #4208:
URL: https://github.com/apache/hadoop/pull/4208
### Description of PR
It is a clean cherry pick from commit 328eae9a146b2dd9857a17a0db6fcddb1de23a0d from trunk.
However, we manually added dependency for assertj. assertj dependency was added as part of a not small patch in HADOOP-16287 and subsequently modified again in HADOOP-16253. Too much overhead in bringing in these PRs and thus we decided to add the dependency manually ourselves.
### How was this patch tested?
mvn test -Dtest=hadoop.io.compress.TestCompressorDecompressor
mvn test -Dtest=hadoop.io.compress.snappy.TestSnappyCompressorDecompressor
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4208: HADOOP-16768. SnappyCompressor test cases wrongly assume that the compressed data is always smaller than the input data.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4208:
URL: https://github.com/apache/hadoop/pull/4208#issuecomment-1104740586
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 39s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 2 new or modified test files. |
|||| _ branch-2.10 Compile Tests _ |
| +0 :ok: | mvndep | 4m 7s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 14m 11s | | branch-2.10 passed |
| +1 :green_heart: | compile | 13m 25s | | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 |
| +1 :green_heart: | compile | 11m 13s | | branch-2.10 passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| +1 :green_heart: | checkstyle | 2m 28s | | branch-2.10 passed |
| +1 :green_heart: | mvnsite | 2m 42s | | branch-2.10 passed |
| +1 :green_heart: | javadoc | 2m 42s | | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 |
| +1 :green_heart: | javadoc | 2m 26s | | branch-2.10 passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| +0 :ok: | spotbugs | 1m 22s | | branch/hadoop-project no spotbugs output file (spotbugsXml.xml) |
| -1 :x: | spotbugs | 2m 24s | [/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4208/2/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html) | hadoop-common-project/hadoop-common in branch-2.10 has 2 extant spotbugs warnings. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 11s | | the patch passed |
| +1 :green_heart: | compile | 12m 26s | | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 |
| +1 :green_heart: | javac | 12m 26s | | the patch passed |
| +1 :green_heart: | compile | 11m 16s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| +1 :green_heart: | javac | 11m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 2m 25s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 40s | | the patch passed |
| +1 :green_heart: | xml | 0m 3s | | The patch has no ill-formed XML file. |
| +1 :green_heart: | javadoc | 2m 39s | | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 |
| +1 :green_heart: | javadoc | 2m 25s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| +0 :ok: | spotbugs | 0m 57s | | hadoop-project has no data from spotbugs |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 55s | | hadoop-project in the patch passed. |
| -1 :x: | unit | 9m 11s | [/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4208/2/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt) | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 1m 16s | | The patch does not generate ASF License warnings. |
| | | 115m 56s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.fs.sftp.TestSFTPFileSystem |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4208/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4208 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml spotbugs checkstyle |
| uname | Linux b28ed3a9ec3a 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-2.10 / 23d7c72915fe7c5dff5178ab5d47366e4b604315 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4208/2/testReport/ |
| Max. process+thread count | 1377 (vs. ulimit of 5500) |
| modules | C: hadoop-project hadoop-common-project/hadoop-common U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4208/2/console |
| versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] xinglin commented on a diff in pull request #4208: HADOOP-16768. SnappyCompressor test cases wrongly assume that the compressed data is always smaller than the input data.
Posted by GitBox <gi...@apache.org>.
xinglin commented on code in PR #4208:
URL: https://github.com/apache/hadoop/pull/4208#discussion_r854824985
##########
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/compress/snappy/TestSnappyCompressorDecompressor.java:
##########
@@ -278,7 +285,38 @@ public void testSnappyBlockCompression() {
fail("testSnappyBlockCompression ex error !!!");
}
}
-
+
+ @Test
+ // The buffer size is smaller than the input.
+ public void testSnappyCompressDecompressWithSmallBuffer() throws Exception {
+ int inputSize = 1024 * 50;
+ int bufferSize = 512;
+ ByteArrayOutputStream out = new ByteArrayOutputStream();
+ byte[] buffer = new byte[bufferSize];
+ byte[] input = BytesGenerator.get(inputSize);
+
+ SnappyCompressor compressor = new SnappyCompressor();
+ compressor.setInput(input, 0, inputSize);
+ compressor.finish();
+ while (!compressor.finished()) {
+ int len = compressor.compress(buffer, 0, buffer.length);
+ out.write(buffer, 0, len);
+ }
+ byte[] compressed = out.toByteArray();
+ assertTrue(compressed.length > 0);
+ out.reset();
+
+ SnappyDecompressor decompressor = new SnappyDecompressor();
+ decompressor.setInput(compressed, 0, compressed.length);
+ while (!decompressor.finished()) {
+ int len = decompressor.decompress(buffer, 0, buffer.length);
+ out.write(buffer, 0, len);
+ }
+ byte[] decompressed = out.toByteArray();
+
+ assertTrue(Arrays.equals(decompressed, input));
Review Comment:
ack. Thanks for the suggestion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4208: HADOOP-16768. SnappyCompressor test cases wrongly assume that the compressed data is always smaller than the input data.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4208:
URL: https://github.com/apache/hadoop/pull/4208#issuecomment-1104641390
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 8m 25s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 2 new or modified test files. |
|||| _ branch-2.10 Compile Tests _ |
| +0 :ok: | mvndep | 4m 7s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 14m 4s | | branch-2.10 passed |
| +1 :green_heart: | compile | 13m 17s | | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 |
| +1 :green_heart: | compile | 11m 15s | | branch-2.10 passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| +1 :green_heart: | checkstyle | 2m 30s | | branch-2.10 passed |
| +1 :green_heart: | mvnsite | 2m 43s | | branch-2.10 passed |
| +1 :green_heart: | javadoc | 2m 46s | | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 |
| +1 :green_heart: | javadoc | 2m 26s | | branch-2.10 passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| +0 :ok: | spotbugs | 1m 25s | | branch/hadoop-project no spotbugs output file (spotbugsXml.xml) |
| -1 :x: | spotbugs | 2m 24s | [/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4208/1/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html) | hadoop-common-project/hadoop-common in branch-2.10 has 2 extant spotbugs warnings. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 10s | | the patch passed |
| -1 :x: | compile | 1m 24s | [/patch-compile-root-jdkAzulSystems,Inc.-1.7.0_262-b10.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4208/1/artifact/out/patch-compile-root-jdkAzulSystems,Inc.-1.7.0_262-b10.txt) | root in the patch failed with JDK Azul Systems, Inc.-1.7.0_262-b10. |
| -1 :x: | javac | 1m 24s | [/patch-compile-root-jdkAzulSystems,Inc.-1.7.0_262-b10.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4208/1/artifact/out/patch-compile-root-jdkAzulSystems,Inc.-1.7.0_262-b10.txt) | root in the patch failed with JDK Azul Systems, Inc.-1.7.0_262-b10. |
| +1 :green_heart: | compile | 10m 41s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| +1 :green_heart: | javac | 10m 41s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 2m 27s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 39s | | the patch passed |
| +1 :green_heart: | xml | 0m 2s | | The patch has no ill-formed XML file. |
| +1 :green_heart: | javadoc | 2m 40s | | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 |
| +1 :green_heart: | javadoc | 2m 26s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| +0 :ok: | spotbugs | 0m 56s | | hadoop-project has no data from spotbugs |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 55s | | hadoop-project in the patch passed. |
| -1 :x: | unit | 9m 11s | [/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4208/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt) | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 1m 17s | | The patch does not generate ASF License warnings. |
| | | 111m 59s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.fs.sftp.TestSFTPFileSystem |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4208/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4208 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml spotbugs checkstyle |
| uname | Linux 69947d45c445 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-2.10 / 647481a3f9b28b25dfe0f5f2400f7a9c6cd22f18 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4208/1/testReport/ |
| Max. process+thread count | 1377 (vs. ulimit of 5500) |
| modules | C: hadoop-project hadoop-common-project/hadoop-common U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4208/1/console |
| versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4208: HADOOP-16768. SnappyCompressor test cases wrongly assume that the compressed data is always smaller than the input data.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #4208:
URL: https://github.com/apache/hadoop/pull/4208#issuecomment-1104867089
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 2 new or modified test files. |
|||| _ branch-2.10 Compile Tests _ |
| +0 :ok: | mvndep | 3m 52s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 14m 12s | | branch-2.10 passed |
| +1 :green_heart: | compile | 13m 36s | | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 |
| +1 :green_heart: | compile | 11m 20s | | branch-2.10 passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| +1 :green_heart: | checkstyle | 2m 29s | | branch-2.10 passed |
| +1 :green_heart: | mvnsite | 2m 45s | | branch-2.10 passed |
| +1 :green_heart: | javadoc | 2m 41s | | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 |
| +1 :green_heart: | javadoc | 2m 22s | | branch-2.10 passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| +0 :ok: | spotbugs | 1m 22s | | branch/hadoop-project no spotbugs output file (spotbugsXml.xml) |
| -1 :x: | spotbugs | 2m 28s | [/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4208/3/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html) | hadoop-common-project/hadoop-common in branch-2.10 has 2 extant spotbugs warnings. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 13s | | the patch passed |
| +1 :green_heart: | compile | 14m 12s | | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 |
| +1 :green_heart: | javac | 14m 12s | | the patch passed |
| +1 :green_heart: | compile | 12m 7s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| +1 :green_heart: | javac | 12m 7s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 2m 27s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 40s | | the patch passed |
| +1 :green_heart: | xml | 0m 2s | | The patch has no ill-formed XML file. |
| +1 :green_heart: | javadoc | 2m 40s | | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 |
| +1 :green_heart: | javadoc | 2m 26s | | the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| +0 :ok: | spotbugs | 0m 57s | | hadoop-project has no data from spotbugs |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 56s | | hadoop-project in the patch passed. |
| -1 :x: | unit | 9m 9s | [/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4208/3/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt) | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 1m 17s | | The patch does not generate ASF License warnings. |
| | | 118m 45s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.fs.sftp.TestSFTPFileSystem |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4208/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/4208 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml spotbugs checkstyle |
| uname | Linux 5ab0817466bc 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-2.10 / 9cc1d785b7548a9cdc8c17fa8bccf138fa749fc9 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4208/3/testReport/ |
| Max. process+thread count | 1378 (vs. ulimit of 5500) |
| modules | C: hadoop-project hadoop-common-project/hadoop-common U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4208/3/console |
| versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] virajith commented on a diff in pull request #4208: HADOOP-16768. SnappyCompressor test cases wrongly assume that the compressed data is always smaller than the input data.
Posted by GitBox <gi...@apache.org>.
virajith commented on code in PR #4208:
URL: https://github.com/apache/hadoop/pull/4208#discussion_r854775362
##########
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/compress/snappy/TestSnappyCompressorDecompressor.java:
##########
@@ -278,7 +285,38 @@ public void testSnappyBlockCompression() {
fail("testSnappyBlockCompression ex error !!!");
}
}
-
+
+ @Test
+ // The buffer size is smaller than the input.
+ public void testSnappyCompressDecompressWithSmallBuffer() throws Exception {
+ int inputSize = 1024 * 50;
+ int bufferSize = 512;
+ ByteArrayOutputStream out = new ByteArrayOutputStream();
+ byte[] buffer = new byte[bufferSize];
+ byte[] input = BytesGenerator.get(inputSize);
+
+ SnappyCompressor compressor = new SnappyCompressor();
+ compressor.setInput(input, 0, inputSize);
+ compressor.finish();
+ while (!compressor.finished()) {
+ int len = compressor.compress(buffer, 0, buffer.length);
+ out.write(buffer, 0, len);
+ }
+ byte[] compressed = out.toByteArray();
+ assertTrue(compressed.length > 0);
+ out.reset();
+
+ SnappyDecompressor decompressor = new SnappyDecompressor();
+ decompressor.setInput(compressed, 0, compressed.length);
+ while (!decompressor.finished()) {
+ int len = decompressor.decompress(buffer, 0, buffer.length);
+ out.write(buffer, 0, len);
+ }
+ byte[] decompressed = out.toByteArray();
+
+ assertTrue(Arrays.equals(decompressed, input));
Review Comment:
you can use assertArrayEquals
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] virajith merged pull request #4208: HADOOP-16768. SnappyCompressor test cases wrongly assume that the compressed data is always smaller than the input data.
Posted by GitBox <gi...@apache.org>.
virajith merged PR #4208:
URL: https://github.com/apache/hadoop/pull/4208
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] virajith commented on pull request #4208: HADOOP-16768. SnappyCompressor test cases wrongly assume that the compressed data is always smaller than the input data.
Posted by GitBox <gi...@apache.org>.
virajith commented on PR #4208:
URL: https://github.com/apache/hadoop/pull/4208#issuecomment-1105292475
Yetus failures unrelated. thanks for the backport @xinglin! Committing this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] xinglin commented on pull request #4208: HADOOP-16768. SnappyCompressor test cases wrongly assume that the compressed data is always smaller than the input data.
Posted by GitBox <gi...@apache.org>.
xinglin commented on PR #4208:
URL: https://github.com/apache/hadoop/pull/4208#issuecomment-1104678805
assertj-core 3.x requires java 8 and thus we can not compile it with jdk 1.7 (java 7) for branch-2.10. Also tried assert-core at 2.9.1 (the latest 2.x release) and it did not compile neither. So, we decided not to use assertj and replaced with AssertTrue/AssertEquals().
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org