You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "EungsopYoo (via GitHub)" <gi...@apache.org> on 2025/05/09 06:31:18 UTC
[PR] Reopen initialReader of HStoreFile to refresh metadata when read failed [hbase]
EungsopYoo opened a new pull request, #6974:
URL: https://github.com/apache/hbase/pull/6974
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29299 Reopen initialReader of HStoreFile to refresh metadata when read failed [hbase]
Posted by "EungsopYoo (via GitHub)" <gi...@apache.org>.
EungsopYoo commented on code in PR #6974:
URL: https://github.com/apache/hbase/pull/6974#discussion_r2101306196
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStoreFile.java:
##########
@@ -514,18 +514,23 @@ private void open() throws IOException {
comparator = initialReader.getComparator();
}
+ public void initReader() throws IOException {
+ initReader(false);
+ }
+
/**
* Initialize the reader used for pread.
*/
- public void initReader() throws IOException {
- if (initialReader == null) {
+ public void initReader(boolean reopen) throws IOException {
+ if (initialReader == null || reopen) {
synchronized (this) {
- if (initialReader == null) {
+ if (initialReader == null || reopen) {
+ boolean evictOnClose = cacheConf == null || cacheConf.shouldEvictOnClose();
Review Comment:
To reuse `evictOnClose` in line 530 and 534.
https://github.com/apache/hbase/pull/6974/files#diff-e62a0bd3a3d8ca0c5cac5cf21711a77fb1c3bec947a32c1645cd8dce3068f2a9R530
https://github.com/apache/hbase/pull/6974/files#diff-e62a0bd3a3d8ca0c5cac5cf21711a77fb1c3bec947a32c1645cd8dce3068f2a9R534
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:
##########
@@ -255,18 +255,14 @@ public StoreScanner(HStore store, ScanInfo scanInfo, Scan scan, NavigableSet<byt
List<KeyValueScanner> scanners = null;
try {
- // Pass columns to try to filter out unnecessary StoreFiles.
- scanners = selectScannersFrom(store,
- store.getScanners(cacheBlocks, scanUsePread, false, matcher, scan.getStartRow(),
- scan.includeStartRow(), scan.getStopRow(), scan.includeStopRow(), this.readPt,
- isOnlyLatestVersionScan(scan)));
-
- // Seek all scanners to the start of the Row (or if the exact matching row
- // key does not exist, then to the start of the next matching Row).
- // Always check bloom filter to optimize the top row seek for delete
- // family marker.
- seekScanners(scanners, matcher.getStartKey(), explicitColumnQuery && lazySeekEnabledGlobally,
- parallelSeekEnabled);
+ try {
+ scanners = getScannersAndSeek(false);
+ } catch (IOException e) {
+ // If we fail to read the store files,
+ // we should close and reopen the store file readers to refresh metadata
+ LOG.info("Reopen StoreScanner", e);
+ scanners = getScannersAndSeek(true);
Review Comment:
This is the only retry mechanism triggered upon failure, so it occurs only under rare conditions. Therefore, I believe it's acceptable to retry on all types of IOException.
However, StoreFileScanner throws IOExceptions with messages that are prefixed with "Could not". So we could use this pattern to decide when to retry. Nevertheless, since these messages may change over time, this approach could become unreliable.
https://github.com/apache/hbase/blob/40b274dee7f49b4dac9df705dfb7267460454ebb/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java#L220
https://github.com/apache/hbase/blob/40b274dee7f49b4dac9df705dfb7267460454ebb/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java#L250
https://github.com/apache/hbase/blob/40b274dee7f49b4dac9df705dfb7267460454ebb/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java#L278
https://github.com/apache/hbase/blob/40b274dee7f49b4dac9df705dfb7267460454ebb/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java#L518
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29299 Reopen initialReader of HStoreFile to refresh metadata when read failed [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #6974:
URL: https://github.com/apache/hbase/pull/6974#issuecomment-2900339494
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 33s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any anti-patterns. |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 17s | | master passed |
| +1 :green_heart: | compile | 3m 10s | | master passed |
| +1 :green_heart: | checkstyle | 0m 36s | | master passed |
| +1 :green_heart: | spotbugs | 1m 41s | | master passed |
| +1 :green_heart: | spotless | 0m 49s | | branch has no errors when running spotless:check. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 2s | | the patch passed |
| +1 :green_heart: | compile | 3m 7s | | the patch passed |
| +1 :green_heart: | javac | 3m 7s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 36s | | the patch passed |
| +1 :green_heart: | spotbugs | 1m 45s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 14m 13s | | Patch does not cause any errors with Hadoop 3.3.6 3.4.0. |
| +1 :green_heart: | spotless | 0m 56s | | patch has no errors when running spotless:check. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 14s | | The patch does not generate ASF License warnings. |
| | | 42m 16s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/6974 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux b2f5746a0190 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 7cf8edfd97ba51b828f7b605ac9d6a6b4258f8b9 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 85 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/3/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29299 Reopen initialReader of HStoreFile to refresh metadata when read failed [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #6974:
URL: https://github.com/apache/hbase/pull/6974#issuecomment-2928269348
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 27s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any anti-patterns. |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 21s | | master passed |
| +1 :green_heart: | compile | 3m 13s | | master passed |
| +1 :green_heart: | checkstyle | 0m 36s | | master passed |
| +1 :green_heart: | spotbugs | 1m 37s | | master passed |
| +1 :green_heart: | spotless | 0m 50s | | branch has no errors when running spotless:check. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 8s | | the patch passed |
| +1 :green_heart: | compile | 3m 10s | | the patch passed |
| +1 :green_heart: | javac | 3m 10s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 38s | | the patch passed |
| +1 :green_heart: | spotbugs | 1m 43s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 12m 16s | | Patch does not cause any errors with Hadoop 3.3.6 3.4.0. |
| +1 :green_heart: | spotless | 0m 45s | | patch has no errors when running spotless:check. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 10s | | The patch does not generate ASF License warnings. |
| | | 39m 48s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/4/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/6974 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux f48de556000a 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ac917157d4e2bb88c36b0bc87f1b4a0f7be89806 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 84 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/4/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29299 Reopen initialReader of HStoreFile to refresh metadata when read failed [hbase]
Posted by "EungsopYoo (via GitHub)" <gi...@apache.org>.
EungsopYoo commented on code in PR #6974:
URL: https://github.com/apache/hbase/pull/6974#discussion_r2101513191
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStoreFile.java:
##########
@@ -514,18 +514,23 @@ private void open() throws IOException {
comparator = initialReader.getComparator();
}
+ public void initReader() throws IOException {
+ initReader(false);
+ }
+
/**
* Initialize the reader used for pread.
*/
- public void initReader() throws IOException {
- if (initialReader == null) {
+ public void initReader(boolean reopen) throws IOException {
+ if (initialReader == null || reopen) {
synchronized (this) {
- if (initialReader == null) {
+ if (initialReader == null || reopen) {
+ boolean evictOnClose = cacheConf == null || cacheConf.shouldEvictOnClose();
try {
+ if (reopen) closeStoreFile(evictOnClose);
Review Comment:
https://github.com/apache/hbase/pull/6974/commits/7cf8edfd97ba51b828f7b605ac9d6a6b4258f8b9
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29299 Reopen initialReader of HStoreFile to refresh metadata when read failed [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #6974:
URL: https://github.com/apache/hbase/pull/6974#issuecomment-2865453716
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 28s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any anti-patterns. |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 21s | | master passed |
| +1 :green_heart: | compile | 3m 8s | | master passed |
| +1 :green_heart: | checkstyle | 0m 37s | | master passed |
| +1 :green_heart: | spotbugs | 1m 35s | | master passed |
| +1 :green_heart: | spotless | 0m 45s | | branch has no errors when running spotless:check. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 3s | | the patch passed |
| +1 :green_heart: | compile | 3m 5s | | the patch passed |
| +1 :green_heart: | javac | 3m 5s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 36s | [/results-checkstyle-hbase-server.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/1/artifact/yetus-general-check/output/results-checkstyle-hbase-server.txt) | hbase-server: The patch generated 1 new + 15 unchanged - 0 fixed = 16 total (was 15) |
| +1 :green_heart: | spotbugs | 1m 41s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 11m 58s | | Patch does not cause any errors with Hadoop 3.3.6 3.4.0. |
| +1 :green_heart: | spotless | 0m 44s | | patch has no errors when running spotless:check. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 10s | | The patch does not generate ASF License warnings. |
| | | 38m 57s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/6974 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux 386672719a0d 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 40b274dee7f49b4dac9df705dfb7267460454ebb |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 84 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/1/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29299 Reopen initialReader of HStoreFile to refresh metadata when read failed [hbase]
Posted by "Apache9 (via GitHub)" <gi...@apache.org>.
Apache9 commented on code in PR #6974:
URL: https://github.com/apache/hbase/pull/6974#discussion_r2119176212
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStoreFile.java:
##########
@@ -514,18 +514,25 @@ private void open() throws IOException {
comparator = initialReader.getComparator();
}
+ public void initReader() throws IOException {
+ initReader(false);
+ }
+
/**
* Initialize the reader used for pread.
*/
- public void initReader() throws IOException {
- if (initialReader == null) {
+ public void initReader(boolean reopen) throws IOException {
+ if (initialReader == null || reopen) {
synchronized (this) {
- if (initialReader == null) {
+ if (initialReader == null || reopen) {
+ boolean evictOnClose = cacheConf == null || cacheConf.shouldEvictOnClose();
try {
+ if (reopen) {
+ closeStoreFile(evictOnClose);
Review Comment:
For reopen, do we really want to evict block cache?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29299 Reopen initialReader of HStoreFile to refresh metadata when read failed [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #6974:
URL: https://github.com/apache/hbase/pull/6974#issuecomment-2928703768
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 27s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
|||| _ Prechecks _ |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 55s | | master passed |
| +1 :green_heart: | compile | 0m 57s | | master passed |
| +1 :green_heart: | javadoc | 0m 28s | | master passed |
| +1 :green_heart: | shadedjars | 6m 6s | | branch has no errors when building our shaded downstream artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 8s | | the patch passed |
| +1 :green_heart: | compile | 0m 57s | | the patch passed |
| +1 :green_heart: | javac | 0m 57s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 27s | | the patch passed |
| +1 :green_heart: | shadedjars | 6m 5s | | patch has no errors when building our shaded downstream artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 211m 35s | | hbase-server in the patch passed. |
| | | 238m 3s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/6974 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux 8aa92d4e8d63 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / ac917157d4e2bb88c36b0bc87f1b4a0f7be89806 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/4/testReport/ |
| Max. process+thread count | 5402 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/4/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29299 Reopen initialReader of HStoreFile to refresh metadata when read failed [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #6974:
URL: https://github.com/apache/hbase/pull/6974#issuecomment-2866076707
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
|||| _ Prechecks _ |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 24s | | master passed |
| +1 :green_heart: | compile | 0m 56s | | master passed |
| +1 :green_heart: | javadoc | 0m 28s | | master passed |
| +1 :green_heart: | shadedjars | 5m 57s | | branch has no errors when building our shaded downstream artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 5s | | the patch passed |
| +1 :green_heart: | compile | 0m 56s | | the patch passed |
| +1 :green_heart: | javac | 0m 56s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 26s | | the patch passed |
| +1 :green_heart: | shadedjars | 5m 54s | | patch has no errors when building our shaded downstream artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 210m 20s | [/patch-unit-hbase-server.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/1/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-server.txt) | hbase-server in the patch failed. |
| | | 235m 56s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/6974 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux aab1dc4a58e0 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 40b274dee7f49b4dac9df705dfb7267460454ebb |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/1/testReport/ |
| Max. process+thread count | 5272 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/1/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29299 Reopen initialReader of HStoreFile to refresh metadata when read failed [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #6974:
URL: https://github.com/apache/hbase/pull/6974#issuecomment-2870276771
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 26s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any anti-patterns. |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 21s | | master passed |
| +1 :green_heart: | compile | 3m 5s | | master passed |
| +1 :green_heart: | checkstyle | 0m 37s | | master passed |
| +1 :green_heart: | spotbugs | 1m 32s | | master passed |
| +1 :green_heart: | spotless | 0m 45s | | branch has no errors when running spotless:check. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 59s | | the patch passed |
| +1 :green_heart: | compile | 3m 1s | | the patch passed |
| +1 :green_heart: | javac | 3m 1s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 36s | [/results-checkstyle-hbase-server.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/2/artifact/yetus-general-check/output/results-checkstyle-hbase-server.txt) | hbase-server: The patch generated 1 new + 16 unchanged - 0 fixed = 17 total (was 16) |
| +1 :green_heart: | spotbugs | 1m 40s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 11m 57s | | Patch does not cause any errors with Hadoop 3.3.6 3.4.0. |
| +1 :green_heart: | spotless | 0m 43s | | patch has no errors when running spotless:check. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 10s | | The patch does not generate ASF License warnings. |
| | | 38m 34s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/6974 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux d586551aaea2 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 297bfc8b661fc36a25115a5e8296633f3e608bf2 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 85 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/2/console |
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29299 Reopen initialReader of HStoreFile to refresh metadata when read failed [hbase]
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #6974:
URL: https://github.com/apache/hbase/pull/6974#issuecomment-2870278662
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 35s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
|||| _ Prechecks _ |
|||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 0s | | master passed |
| +1 :green_heart: | compile | 1m 18s | | master passed |
| +1 :green_heart: | javadoc | 0m 39s | | master passed |
| +1 :green_heart: | shadedjars | 6m 46s | | branch has no errors when building our shaded downstream artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 52s | | the patch passed |
| +1 :green_heart: | compile | 1m 8s | | the patch passed |
| +1 :green_heart: | javac | 1m 8s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed |
| +1 :green_heart: | shadedjars | 6m 41s | | patch has no errors when building our shaded downstream artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 15m 52s | [/patch-unit-hbase-server.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/2/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-server.txt) | hbase-server in the patch failed. |
| | | 43m 12s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/6974 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux 4a4eb04f6fc1 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 297bfc8b661fc36a25115a5e8296633f3e608bf2 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/2/testReport/ |
| Max. process+thread count | 1854 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6974/2/console |
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29299 Reopen initialReader of HStoreFile to refresh metadata when read failed [hbase]
Posted by "EungsopYoo (via GitHub)" <gi...@apache.org>.
EungsopYoo commented on code in PR #6974:
URL: https://github.com/apache/hbase/pull/6974#discussion_r2119768764
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStoreFile.java:
##########
@@ -514,18 +514,25 @@ private void open() throws IOException {
comparator = initialReader.getComparator();
}
+ public void initReader() throws IOException {
+ initReader(false);
+ }
+
/**
* Initialize the reader used for pread.
*/
- public void initReader() throws IOException {
- if (initialReader == null) {
+ public void initReader(boolean reopen) throws IOException {
+ if (initialReader == null || reopen) {
synchronized (this) {
- if (initialReader == null) {
+ if (initialReader == null || reopen) {
+ boolean evictOnClose = cacheConf == null || cacheConf.shouldEvictOnClose();
try {
+ if (reopen) {
+ closeStoreFile(evictOnClose);
Review Comment:
https://github.com/apache/hbase/pull/6974/commits/ac917157d4e2bb88c36b0bc87f1b4a0f7be89806
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29299 Reopen initialReader of HStoreFile to refresh metadata when read failed [hbase]
Posted by "Apache9 (via GitHub)" <gi...@apache.org>.
Apache9 commented on code in PR #6974:
URL: https://github.com/apache/hbase/pull/6974#discussion_r2101494002
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStoreFile.java:
##########
@@ -514,18 +514,23 @@ private void open() throws IOException {
comparator = initialReader.getComparator();
}
+ public void initReader() throws IOException {
+ initReader(false);
+ }
+
/**
* Initialize the reader used for pread.
*/
- public void initReader() throws IOException {
- if (initialReader == null) {
+ public void initReader(boolean reopen) throws IOException {
+ if (initialReader == null || reopen) {
synchronized (this) {
- if (initialReader == null) {
+ if (initialReader == null || reopen) {
+ boolean evictOnClose = cacheConf == null || cacheConf.shouldEvictOnClose();
try {
+ if (reopen) closeStoreFile(evictOnClose);
Review Comment:
OK, we use evictOnClose here too.
Please use change to
```
if () {
}
```
To avoid a checkstyle warning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] HBASE-29299 Reopen initialReader of HStoreFile to refresh metadata when read failed [hbase]
Posted by "Apache9 (via GitHub)" <gi...@apache.org>.
Apache9 commented on code in PR #6974:
URL: https://github.com/apache/hbase/pull/6974#discussion_r2100537785
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStoreFile.java:
##########
@@ -514,18 +514,23 @@ private void open() throws IOException {
comparator = initialReader.getComparator();
}
+ public void initReader() throws IOException {
+ initReader(false);
+ }
+
/**
* Initialize the reader used for pread.
*/
- public void initReader() throws IOException {
- if (initialReader == null) {
+ public void initReader(boolean reopen) throws IOException {
+ if (initialReader == null || reopen) {
synchronized (this) {
- if (initialReader == null) {
+ if (initialReader == null || reopen) {
+ boolean evictOnClose = cacheConf == null || cacheConf.shouldEvictOnClose();
Review Comment:
Why move this line here?
##########
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:
##########
@@ -255,18 +255,14 @@ public StoreScanner(HStore store, ScanInfo scanInfo, Scan scan, NavigableSet<byt
List<KeyValueScanner> scanners = null;
try {
- // Pass columns to try to filter out unnecessary StoreFiles.
- scanners = selectScannersFrom(store,
- store.getScanners(cacheBlocks, scanUsePread, false, matcher, scan.getStartRow(),
- scan.includeStartRow(), scan.getStopRow(), scan.includeStopRow(), this.readPt,
- isOnlyLatestVersionScan(scan)));
-
- // Seek all scanners to the start of the Row (or if the exact matching row
- // key does not exist, then to the start of the next matching Row).
- // Always check bloom filter to optimize the top row seek for delete
- // family marker.
- seekScanners(scanners, matcher.getStartKey(), explicitColumnQuery && lazySeekEnabledGlobally,
- parallelSeekEnabled);
+ try {
+ scanners = getScannersAndSeek(false);
+ } catch (IOException e) {
+ // If we fail to read the store files,
+ // we should close and reopen the store file readers to refresh metadata
+ LOG.info("Reopen StoreScanner", e);
+ scanners = getScannersAndSeek(true);
Review Comment:
We want to retry for all types of IOException? Not some specific ones?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org