You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/02/20 18:10:26 UTC
[GitHub] [hbase] virajjasani opened a new pull request #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin…
virajjasani opened a new pull request #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin…
URL: https://github.com/apache/hbase/pull/1187
…g is enabled (Backport HBASE-23066)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] virajjasani edited a comment on issue #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin…
Posted by GitBox <gi...@apache.org>.
virajjasani edited a comment on issue #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin…
URL: https://github.com/apache/hbase/pull/1187#issuecomment-590827126
FYI @ramkrish86 @chenxu14 in case Reid's question is discussed before in parent or other relevant Jira. It is interesting and I don't have data/ans from perf POV.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] virajjasani commented on issue #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin…
Posted by GitBox <gi...@apache.org>.
virajjasani commented on issue #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin…
URL: https://github.com/apache/hbase/pull/1187#issuecomment-590827126
FYI @ramkrish86 @chenxu14 in case Reid's question is discussed before in parent or other relevant Jira. It is interesting and I don't have data from perf POV.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] ramkrish86 commented on issue #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin…
Posted by GitBox <gi...@apache.org>.
ramkrish86 commented on issue #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin…
URL: https://github.com/apache/hbase/pull/1187#issuecomment-590990341
@Reidddddd - I think what you mean is that if blocks are cached on compaction is there any impact the cache usage ? Ya it is. Since this is aggressive caching probably we need bigger caches avialable for this feature to work. Say for eg if cacheOnWrite is also enabled then blocks are cached during flushes and also during compactions. So at any point of time there could be duplicate blocks in the cache thus bloating the cache. LRU will remove the old blocks anyway but still the usage is going to be more. but in clould usages (as you can see in parent JIRa) a compaction will clear all the blocks from the cache. So any scans that happens after a compaction will have high latencies. So in such cases with bigger caches this new config will help to have the blocks in cache and thus having more predictable scan latencies. Hence it was later decided (if you see the subjiras in the parent JIRA) that lets cache the index blocks also if this property and cacheOnWrite is set to true.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] virajjasani closed pull request #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin…
Posted by GitBox <gi...@apache.org>.
virajjasani closed pull request #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin…
URL: https://github.com/apache/hbase/pull/1187
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Reidddddd commented on issue #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin…
Posted by GitBox <gi...@apache.org>.
Reidddddd commented on issue #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin…
URL: https://github.com/apache/hbase/pull/1187#issuecomment-590816361
LGTM. May I ask what is the consequence of table is big and cachecompactedblocksonwrite is on.
(Sorry. I lack some background information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin…
URL: https://github.com/apache/hbase/pull/1187#issuecomment-589321670
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 7m 22s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ branch-1 Compile Tests _ |
| +1 :green_heart: | mvninstall | 8m 36s | branch-1 passed |
| +1 :green_heart: | compile | 0m 41s | branch-1 passed with JDK v1.8.0_242 |
| +1 :green_heart: | compile | 0m 43s | branch-1 passed with JDK v1.7.0_252 |
| +1 :green_heart: | checkstyle | 1m 39s | branch-1 passed |
| +1 :green_heart: | shadedjars | 2m 51s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 41s | branch-1 passed with JDK v1.8.0_242 |
| +1 :green_heart: | javadoc | 0m 41s | branch-1 passed with JDK v1.7.0_252 |
| +0 :ok: | spotbugs | 2m 48s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 44s | branch-1 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 54s | the patch passed |
| +1 :green_heart: | compile | 0m 41s | the patch passed with JDK v1.8.0_242 |
| +1 :green_heart: | javac | 0m 41s | the patch passed |
| +1 :green_heart: | compile | 0m 44s | the patch passed with JDK v1.7.0_252 |
| +1 :green_heart: | javac | 0m 44s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 34s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedjars | 2m 47s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 4m 55s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. |
| +1 :green_heart: | javadoc | 0m 32s | the patch passed with JDK v1.8.0_242 |
| +1 :green_heart: | javadoc | 0m 40s | the patch passed with JDK v1.7.0_252 |
| +1 :green_heart: | findbugs | 2m 53s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 127m 22s | hbase-server in the patch failed. |
| +1 :green_heart: | asflicense | 0m 34s | The patch does not generate ASF License warnings. |
| | | 174m 13s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1187/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1187 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux fb7714f9c213 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1187/out/precommit/personality/provided.sh |
| git revision | branch-1 / 25654df |
| Default Java | 1.7.0_252 |
| Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_242 /usr/lib/jvm/zulu-7-amd64:1.7.0_252 |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1187/1/artifact/out/patch-unit-hbase-server.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1187/1/testReport/ |
| Max. process+thread count | 4428 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1187/1/console |
| versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services