You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "charlesconnell (via GitHub)" <gi...@apache.org> on 2025/03/07 03:02:06 UTC
[PR] HBASE-29172: Fix to bug in ZstdByteBuffDecompressor [hbase]
charlesconnell opened a new pull request, #6767:
URL: https://github.com/apache/hbase/pull/6767
My recent contribution in HBASE-29135 has a bug. `ZstdByteBuffDecompressor` doesn't update the position of the input `ByteBuffer` after reading from it. When there are multiple "chunks" in the input buffer, this causes failure to read chunks after the first one.
This PR fixes the bug and adjusts an existing test to verify the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29172: Fix to bug in ZstdByteBuffDecompressor [hbase]
Posted by "ndimiduk (via GitHub)" <gi...@apache.org>.
ndimiduk commented on code in PR #6767:
URL: https://github.com/apache/hbase/pull/6767#discussion_r1987625121
##########
hbase-compression/hbase-compression-zstd/src/test/java/org/apache/hadoop/hbase/io/compress/zstd/TestZstdByteBuffDecompressor.java:
##########
@@ -40,10 +40,14 @@ public class TestZstdByteBuffDecompressor {
public static final HBaseClassTestRule CLASS_RULE =
HBaseClassTestRule.forClass(TestZstdByteBuffDecompressor.class);
- // "HBase is awesome" compressed with zstd, and then prepended with metadata as a
- // BlockCompressorStream would
- private static final byte[] COMPRESSED_PAYLOAD =
- Bytes.fromHex("000000100000001928b52ffd2010810000484261736520697320617765736f6d65");
+ /*
+ * "HBase is fun to use and very fast" compressed with zstd, and then prepended with metadata as a
+ * BlockCompressorStream would. The phrase is split in three parts and put into the payload in
+ * this structure: (block 1: (chunk 1: HBase is), (chunk 2: fun to use)), (block 2: (chunk 1: and
+ * very fast))
+ */
+ private static final byte[] COMPRESSED_PAYLOAD = Bytes.fromHex(
+ "000000130000001228b52ffd20094900004842617365206973200000001428b52ffd200b59000066756e20746f20757365200000000d0000001628b52ffd200d690000616e6420766572792066617374");
Review Comment:
Yeah, might as well split the long line. Then someone else won't have to clean it up later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29172: Fix to bug in ZstdByteBuffDecompressor [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #6767:
URL: https://github.com/apache/hbase/pull/6767#issuecomment-2711998223
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 1m 0s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any anti-patterns. |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 46s | | master passed |
| +1 :green_heart: | compile | 0m 40s | | master passed |
| +1 :green_heart: | checkstyle | 0m 15s | | master passed |
| +1 :green_heart: | spotbugs | 0m 33s | | master passed |
| +1 :green_heart: | spotless | 1m 6s | | branch has no errors when running spotless:check. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 21s | | the patch passed |
| +1 :green_heart: | compile | 0m 19s | | the patch passed |
| +1 :green_heart: | javac | 0m 19s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 7s | | the patch passed |
| +1 :green_heart: | spotbugs | 0m 26s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 12m 4s | | Patch does not cause any errors with Hadoop 3.3.6 3.4.0. |
| +1 :green_heart: | spotless | 0m 45s | | patch has no errors when running spotless:check. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 10s | | The patch does not generate ASF License warnings. |
| | | 34m 17s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6767/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/6767 |
| JIRA Issue | HBASE-29172 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux 40bc9c6c5470 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ae7fbf683a9bec6366c3e0880da66d27c8e97aa7 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 83 (vs. ulimit of 30000) |
| modules | C: hbase-compression/hbase-compression-zstd U: hbase-compression/hbase-compression-zstd |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6767/3/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29172: Fix to bug in ZstdByteBuffDecompressor [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #6767:
URL: https://github.com/apache/hbase/pull/6767#issuecomment-2706420610
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 32s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
|||| _ Prechecks _ |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 34s | | master passed |
| +1 :green_heart: | compile | 0m 17s | | master passed |
| +1 :green_heart: | javadoc | 0m 14s | | master passed |
| +1 :green_heart: | shadedjars | 7m 2s | | branch has no errors when building our shaded downstream artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 11s | | the patch passed |
| +1 :green_heart: | compile | 0m 20s | | the patch passed |
| +1 :green_heart: | javac | 0m 20s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 14s | | the patch passed |
| +1 :green_heart: | shadedjars | 6m 59s | | patch has no errors when building our shaded downstream artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 4m 51s | | hbase-compression-zstd in the patch passed. |
| | | 29m 18s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6767/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/6767 |
| JIRA Issue | HBASE-29172 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux 1c67567c818a 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 5f99b76abf6b2d26cedca014b50d25dd90b07a55 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6767/2/testReport/ |
| Max. process+thread count | 609 (vs. ulimit of 30000) |
| modules | C: hbase-compression/hbase-compression-zstd U: hbase-compression/hbase-compression-zstd |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6767/2/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29172: Fix to bug in ZstdByteBuffDecompressor [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #6767:
URL: https://github.com/apache/hbase/pull/6767#issuecomment-2705451321
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 26s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
|||| _ Prechecks _ |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 16s | | master passed |
| +1 :green_heart: | compile | 0m 14s | | master passed |
| +1 :green_heart: | javadoc | 0m 13s | | master passed |
| +1 :green_heart: | shadedjars | 5m 52s | | branch has no errors when building our shaded downstream artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 0s | | the patch passed |
| +1 :green_heart: | compile | 0m 15s | | the patch passed |
| +1 :green_heart: | javac | 0m 15s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 11s | | the patch passed |
| +1 :green_heart: | shadedjars | 5m 50s | | patch has no errors when building our shaded downstream artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 3m 48s | | hbase-compression-zstd in the patch passed. |
| | | 24m 2s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6767/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/6767 |
| JIRA Issue | HBASE-29172 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux 33ded455e47d 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 5e01d34b5adf92233e3fa72d9b7f06c05fd7810b |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6767/1/testReport/ |
| Max. process+thread count | 522 (vs. ulimit of 30000) |
| modules | C: hbase-compression/hbase-compression-zstd U: hbase-compression/hbase-compression-zstd |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6767/1/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29172: Fix to bug in ZstdByteBuffDecompressor [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #6767:
URL: https://github.com/apache/hbase/pull/6767#issuecomment-2705455967
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 27s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any anti-patterns. |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 17s | | master passed |
| +1 :green_heart: | compile | 0m 24s | | master passed |
| +1 :green_heart: | checkstyle | 0m 9s | | master passed |
| +1 :green_heart: | spotbugs | 0m 22s | | master passed |
| +1 :green_heart: | spotless | 0m 45s | | branch has no errors when running spotless:check. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 0s | | the patch passed |
| +1 :green_heart: | compile | 0m 18s | | the patch passed |
| +1 :green_heart: | javac | 0m 18s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 7s | [/results-checkstyle-hbase-compression_hbase-compression-zstd.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6767/1/artifact/yetus-general-check/output/results-checkstyle-hbase-compression_hbase-compression-zstd.txt) | hbase-compression/hbase-compression-zstd: The patch generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) |
| +1 :green_heart: | spotbugs | 0m 26s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 11m 47s | | Patch does not cause any errors with Hadoop 3.3.6 3.4.0. |
| -1 :x: | spotless | 0m 43s | | patch has 55 errors when running spotless:check, run spotless:apply to fix. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 9s | | The patch does not generate ASF License warnings. |
| | | 29m 21s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6767/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/6767 |
| JIRA Issue | HBASE-29172 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux d18390dfec92 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 5e01d34b5adf92233e3fa72d9b7f06c05fd7810b |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6767/1/artifact/yetus-general-check/output/patch-spotless.txt |
| Max. process+thread count | 83 (vs. ulimit of 30000) |
| modules | C: hbase-compression/hbase-compression-zstd U: hbase-compression/hbase-compression-zstd |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6767/1/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29172: Fix to bug in ZstdByteBuffDecompressor [hbase]
Posted by "charlesconnell (via GitHub)" <gi...@apache.org>.
charlesconnell commented on code in PR #6767:
URL: https://github.com/apache/hbase/pull/6767#discussion_r1987528201
##########
hbase-compression/hbase-compression-zstd/src/test/java/org/apache/hadoop/hbase/io/compress/zstd/TestZstdByteBuffDecompressor.java:
##########
@@ -40,10 +40,14 @@ public class TestZstdByteBuffDecompressor {
public static final HBaseClassTestRule CLASS_RULE =
HBaseClassTestRule.forClass(TestZstdByteBuffDecompressor.class);
- // "HBase is awesome" compressed with zstd, and then prepended with metadata as a
- // BlockCompressorStream would
- private static final byte[] COMPRESSED_PAYLOAD =
- Bytes.fromHex("000000100000001928b52ffd2010810000484261736520697320617765736f6d65");
+ /*
+ * "HBase is fun to use and very fast" compressed with zstd, and then prepended with metadata as a
+ * BlockCompressorStream would. The phrase is split in three parts and put into the payload in
+ * this structure: (block 1: (chunk 1: HBase is), (chunk 2: fun to use)), (block 2: (chunk 1: and
+ * very fast))
+ */
+ private static final byte[] COMPRESSED_PAYLOAD = Bytes.fromHex(
+ "000000130000001228b52ffd20094900004842617365206973200000001428b52ffd200b59000066756e20746f20757365200000000d0000001628b52ffd200d690000616e6420766572792066617374");
Review Comment:
tbh if spotless is okay with this, I am too
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29172: Fix to bug in ZstdByteBuffDecompressor [hbase]
Posted by "ndimiduk (via GitHub)" <gi...@apache.org>.
ndimiduk merged PR #6767:
URL: https://github.com/apache/hbase/pull/6767
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29172: Fix to bug in ZstdByteBuffDecompressor [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #6767:
URL: https://github.com/apache/hbase/pull/6767#issuecomment-2706420870
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 33s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any anti-patterns. |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 15s | | master passed |
| +1 :green_heart: | compile | 0m 19s | | master passed |
| +1 :green_heart: | checkstyle | 0m 8s | | master passed |
| +1 :green_heart: | spotbugs | 0m 21s | | master passed |
| +1 :green_heart: | spotless | 0m 45s | | branch has no errors when running spotless:check. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 2s | | the patch passed |
| +1 :green_heart: | compile | 0m 18s | | the patch passed |
| +1 :green_heart: | javac | 0m 18s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 7s | [/results-checkstyle-hbase-compression_hbase-compression-zstd.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6767/2/artifact/yetus-general-check/output/results-checkstyle-hbase-compression_hbase-compression-zstd.txt) | hbase-compression/hbase-compression-zstd: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | spotbugs | 0m 26s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 11m 54s | | Patch does not cause any errors with Hadoop 3.3.6 3.4.0. |
| +1 :green_heart: | spotless | 0m 43s | | patch has no errors when running spotless:check. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 9s | | The patch does not generate ASF License warnings. |
| | | 29m 28s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6767/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/6767 |
| JIRA Issue | HBASE-29172 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux 77895ce9695f 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 5f99b76abf6b2d26cedca014b50d25dd90b07a55 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 81 (vs. ulimit of 30000) |
| modules | C: hbase-compression/hbase-compression-zstd U: hbase-compression/hbase-compression-zstd |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6767/2/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29172: Fix to bug in ZstdByteBuffDecompressor [hbase]
Posted by "rmdmattingly (via GitHub)" <gi...@apache.org>.
rmdmattingly commented on code in PR #6767:
URL: https://github.com/apache/hbase/pull/6767#discussion_r1987523249
##########
hbase-compression/hbase-compression-zstd/src/test/java/org/apache/hadoop/hbase/io/compress/zstd/TestZstdByteBuffDecompressor.java:
##########
@@ -40,10 +40,14 @@ public class TestZstdByteBuffDecompressor {
public static final HBaseClassTestRule CLASS_RULE =
HBaseClassTestRule.forClass(TestZstdByteBuffDecompressor.class);
- // "HBase is awesome" compressed with zstd, and then prepended with metadata as a
- // BlockCompressorStream would
- private static final byte[] COMPRESSED_PAYLOAD =
- Bytes.fromHex("000000100000001928b52ffd2010810000484261736520697320617765736f6d65");
+ /*
+ * "HBase is fun to use and very fast" compressed with zstd, and then prepended with metadata as a
+ * BlockCompressorStream would. The phrase is split in three parts and put into the payload in
+ * this structure: (block 1: (chunk 1: HBase is), (chunk 2: fun to use)), (block 2: (chunk 1: and
+ * very fast))
+ */
+ private static final byte[] COMPRESSED_PAYLOAD = Bytes.fromHex(
+ "000000130000001228b52ffd20094900004842617365206973200000001428b52ffd200b59000066756e20746f20757365200000000d0000001628b52ffd200d690000616e6420766572792066617374");
Review Comment:
nit: should we break this up to appease checkstyle?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29172: Fix to bug in ZstdByteBuffDecompressor [hbase]
Posted by "charlesconnell (via GitHub)" <gi...@apache.org>.
charlesconnell commented on code in PR #6767:
URL: https://github.com/apache/hbase/pull/6767#discussion_r1987723309
##########
hbase-compression/hbase-compression-zstd/src/test/java/org/apache/hadoop/hbase/io/compress/zstd/TestZstdByteBuffDecompressor.java:
##########
@@ -40,10 +40,14 @@ public class TestZstdByteBuffDecompressor {
public static final HBaseClassTestRule CLASS_RULE =
HBaseClassTestRule.forClass(TestZstdByteBuffDecompressor.class);
- // "HBase is awesome" compressed with zstd, and then prepended with metadata as a
- // BlockCompressorStream would
- private static final byte[] COMPRESSED_PAYLOAD =
- Bytes.fromHex("000000100000001928b52ffd2010810000484261736520697320617765736f6d65");
+ /*
+ * "HBase is fun to use and very fast" compressed with zstd, and then prepended with metadata as a
+ * BlockCompressorStream would. The phrase is split in three parts and put into the payload in
+ * this structure: (block 1: (chunk 1: HBase is), (chunk 2: fun to use)), (block 2: (chunk 1: and
+ * very fast))
+ */
+ private static final byte[] COMPRESSED_PAYLOAD = Bytes.fromHex(
+ "000000130000001228b52ffd20094900004842617365206973200000001428b52ffd200b59000066756e20746f20757365200000000d0000001628b52ffd200d690000616e6420766572792066617374");
Review Comment:
No problem, done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29172: Fix to bug in ZstdByteBuffDecompressor [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #6767:
URL: https://github.com/apache/hbase/pull/6767#issuecomment-2711992420
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 33s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
|||| _ Prechecks _ |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 9s | | master passed |
| +1 :green_heart: | compile | 0m 22s | | master passed |
| +1 :green_heart: | javadoc | 0m 16s | | master passed |
| +1 :green_heart: | shadedjars | 7m 23s | | branch has no errors when building our shaded downstream artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 58s | | the patch passed |
| +1 :green_heart: | compile | 0m 21s | | the patch passed |
| +1 :green_heart: | javac | 0m 21s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 16s | | the patch passed |
| +1 :green_heart: | shadedjars | 7m 3s | | patch has no errors when building our shaded downstream artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 4m 51s | | hbase-compression-zstd in the patch passed. |
| | | 30m 12s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6767/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/6767 |
| JIRA Issue | HBASE-29172 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux a7965d1f5c5c 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ae7fbf683a9bec6366c3e0880da66d27c8e97aa7 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6767/3/testReport/ |
| Max. process+thread count | 603 (vs. ulimit of 30000) |
| modules | C: hbase-compression/hbase-compression-zstd U: hbase-compression/hbase-compression-zstd |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6767/3/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org