You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2023/01/10 13:33:12 UTC
[GitHub] [hbase] ndimiduk opened a new pull request, #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
ndimiduk opened a new pull request, #4954:
URL: https://github.com/apache/hbase/pull/4954
Clamp the boundaries of the selected sublist according to the input list size.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
Apache9 commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1378900636
Checked the code, the ratio is percentage of regionservers we want to restart, normally, so in general it should be a value between 0 and 1. But there is no sanity check in our code...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a diff in pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
Apache9 commented on code in PR #4954:
URL: https://github.com/apache/hbase/pull/4954#discussion_r1067756498
##########
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/monkies/PolicyBasedChaosMonkey.java:
##########
@@ -116,13 +119,16 @@ public static <T> T selectWeightedRandomItem(List<Pair<T, Integer>> items) {
/** Selects and returns ceil(ratio * items.length) random items from the given array */
public static <T> List<T> selectRandomItems(T[] items, float ratio) {
- int selectedNumber = (int) Math.ceil(items.length * ratio);
-
- List<T> originalItems = Arrays.asList(items);
+ final int selectedNumber = (int) Math.ceil(items.length * ratio);
+ final List<T> originalItems = Arrays.asList(items);
Review Comment:
Arrays.asList will create a ArrayList which is backed by the given array, so the later shuffle will change the order of the items array which is passed in as a parameter. If we think this is acceptable, we can just shuffle on the given array instead of wrapping it? If not, I think we should copy the array.
##########
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/monkies/PolicyBasedChaosMonkey.java:
##########
@@ -116,13 +119,16 @@ public static <T> T selectWeightedRandomItem(List<Pair<T, Integer>> items) {
/** Selects and returns ceil(ratio * items.length) random items from the given array */
public static <T> List<T> selectRandomItems(T[] items, float ratio) {
- int selectedNumber = (int) Math.ceil(items.length * ratio);
-
- List<T> originalItems = Arrays.asList(items);
+ final int selectedNumber = (int) Math.ceil(items.length * ratio);
+ final List<T> originalItems = Arrays.asList(items);
Collections.shuffle(originalItems);
-
- int startIndex = ThreadLocalRandom.current().nextInt(items.length - selectedNumber);
- return originalItems.subList(startIndex, startIndex + selectedNumber);
+ if (selectedNumber == items.length) {
Review Comment:
Use >= for safety?
##########
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/monkies/PolicyBasedChaosMonkey.java:
##########
@@ -116,13 +116,13 @@ public static <T> T selectWeightedRandomItem(List<Pair<T, Integer>> items) {
/** Selects and returns ceil(ratio * items.length) random items from the given array */
public static <T> List<T> selectRandomItems(T[] items, float ratio) {
- int selectedNumber = (int) Math.ceil(items.length * ratio);
-
- List<T> originalItems = Arrays.asList(items);
+ final int selectedNumber = (int) Math.ceil(items.length * ratio);
+ final List<T> originalItems = Arrays.asList(items);
Collections.shuffle(originalItems);
-
- int startIndex = ThreadLocalRandom.current().nextInt(items.length - selectedNumber);
- return originalItems.subList(startIndex, startIndex + selectedNumber);
+ final int startIndex =
Review Comment:
I think we'd better add some comments here to say why we need these guards? Is it because of the floating point precision?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a diff in pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
Apache9 commented on code in PR #4954:
URL: https://github.com/apache/hbase/pull/4954#discussion_r1066543958
##########
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/monkies/PolicyBasedChaosMonkey.java:
##########
@@ -116,13 +116,13 @@ public static <T> T selectWeightedRandomItem(List<Pair<T, Integer>> items) {
/** Selects and returns ceil(ratio * items.length) random items from the given array */
public static <T> List<T> selectRandomItems(T[] items, float ratio) {
- int selectedNumber = (int) Math.ceil(items.length * ratio);
-
- List<T> originalItems = Arrays.asList(items);
+ final int selectedNumber = (int) Math.ceil(items.length * ratio);
+ final List<T> originalItems = Arrays.asList(items);
Collections.shuffle(originalItems);
-
- int startIndex = ThreadLocalRandom.current().nextInt(items.length - selectedNumber);
- return originalItems.subList(startIndex, startIndex + selectedNumber);
+ final int startIndex =
Review Comment:
So the problem here is ratio could be greater than 1.0?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a diff in pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on code in PR #4954:
URL: https://github.com/apache/hbase/pull/4954#discussion_r1067043473
##########
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/monkies/PolicyBasedChaosMonkey.java:
##########
@@ -116,13 +116,13 @@ public static <T> T selectWeightedRandomItem(List<Pair<T, Integer>> items) {
/** Selects and returns ceil(ratio * items.length) random items from the given array */
public static <T> List<T> selectRandomItems(T[] items, float ratio) {
- int selectedNumber = (int) Math.ceil(items.length * ratio);
-
- List<T> originalItems = Arrays.asList(items);
+ final int selectedNumber = (int) Math.ceil(items.length * ratio);
+ final List<T> originalItems = Arrays.asList(items);
Collections.shuffle(originalItems);
-
- int startIndex = ThreadLocalRandom.current().nextInt(items.length - selectedNumber);
- return originalItems.subList(startIndex, startIndex + selectedNumber);
+ final int startIndex =
Review Comment:
I think it's more likely that I have `ratio == length`, we'll have this problem... Let me add some debugging and see what I can learn.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a diff in pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
Apache9 commented on code in PR #4954:
URL: https://github.com/apache/hbase/pull/4954#discussion_r1068074857
##########
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/monkies/PolicyBasedChaosMonkey.java:
##########
@@ -116,13 +120,31 @@ public static <T> T selectWeightedRandomItem(List<Pair<T, Integer>> items) {
/** Selects and returns ceil(ratio * items.length) random items from the given array */
public static <T> List<T> selectRandomItems(T[] items, float ratio) {
- int selectedNumber = (int) Math.ceil(items.length * ratio);
+ /*
+ * N.b. `ratio` values are not validated. Be aware of excessive values and floating point
+ * arithmetic rounding. Guard against negative input to Random#next() and exceeding boundaries
+ * in call to List#subList.
+ */
- List<T> originalItems = Arrays.asList(items);
- Collections.shuffle(originalItems);
+ // clamp ratio to [0.0,1.0]
+ ratio = Math.max(Math.min(ratio, 1.0f), 0.0f);
- int startIndex = ThreadLocalRandom.current().nextInt(items.length - selectedNumber);
- return originalItems.subList(startIndex, startIndex + selectedNumber);
+ final int selectedNumber = (int) Math.ceil(items.length * ratio);
+
+ // shuffle a copy of the input, not the input.
+ final List<T> shuffledItems = new ArrayList<>(items.length);
+ shuffledItems.addAll(Arrays.asList(items));
+ Collections.shuffle(shuffledItems);
+
+ if (selectedNumber >= items.length) {
+ return shuffledItems;
+ }
+
+ // apply basic sanity check on sublist selection range.
+ final int startIndex =
+ Math.max(0, ThreadLocalRandom.current().nextInt(items.length - selectedNumber));
+ final int endIndex = Math.min(items.length, startIndex + selectedNumber);
+ return shuffledItems.subList(startIndex, endIndex);
Review Comment:
Maybe here you could make use of org.apache.hadoop.hbase.util.ReservoirSample so we do not need to copy the whole array? Not a blocker issue anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1378984679
```
2023-01-11T15:39:46,370 DEBUG [ChaosMonkey-0] monkies.PolicyBasedChaosMonkey: selectRandomItems(10, 1.0) of type class [Lorg.apache.hadoop.hbase.ServerName;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1380235327
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 17s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 46s | master passed |
| +1 :green_heart: | compile | 0m 31s | master passed |
| +1 :green_heart: | shadedjars | 5m 0s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 14s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 38s | the patch passed |
| +1 :green_heart: | compile | 0m 18s | the patch passed |
| +1 :green_heart: | javac | 0m 18s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 12s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 10s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 39s | hbase-it in the patch passed. |
| | | 19m 47s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4954 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 9ba215d0dbc5 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / c1b9d8ba06 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/3/testReport/ |
| Max. process+thread count | 582 (vs. ulimit of 30000) |
| modules | C: hbase-it U: hbase-it |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/3/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1380564325
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 21s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 2s | master passed |
| +1 :green_heart: | compile | 0m 31s | master passed |
| +1 :green_heart: | checkstyle | 0m 12s | master passed |
| +1 :green_heart: | spotless | 0m 48s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 0m 28s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 11s | the patch passed |
| +1 :green_heart: | compile | 0m 38s | the patch passed |
| +1 :green_heart: | javac | 0m 38s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 12s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 12m 8s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. |
| +1 :green_heart: | spotless | 0m 55s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 0m 38s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 9s | The patch does not generate ASF License warnings. |
| | | 32m 8s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/5/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4954 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux b65c4417fd20 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / dff8e50849 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Max. process+thread count | 84 (vs. ulimit of 30000) |
| modules | C: hbase-it U: hbase-it |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/5/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1377329969
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 10s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 49s | master passed |
| +1 :green_heart: | compile | 0m 29s | master passed |
| +1 :green_heart: | checkstyle | 0m 13s | master passed |
| +1 :green_heart: | spotless | 0m 43s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 0m 26s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 33s | the patch passed |
| +1 :green_heart: | compile | 0m 27s | the patch passed |
| +1 :green_heart: | javac | 0m 27s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 12s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 9m 50s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. |
| +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 0m 33s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. |
| | | 26m 29s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4954 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux 97fb5422e683 5.4.0-131-generic #147-Ubuntu SMP Fri Oct 14 17:07:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 4add5250ed |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Max. process+thread count | 79 (vs. ulimit of 30000) |
| modules | C: hbase-it U: hbase-it |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/1/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1380549479
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 38s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 2s | master passed |
| +1 :green_heart: | compile | 0m 15s | master passed |
| +1 :green_heart: | shadedjars | 10m 12s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 10s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 45s | the patch passed |
| +1 :green_heart: | compile | 0m 15s | the patch passed |
| +1 :green_heart: | javac | 0m 15s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 11s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 11s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 43s | hbase-it in the patch passed. |
| | | 22m 29s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4954 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 9f8ea032e44e 5.4.0-1088-aws #96~18.04.1-Ubuntu SMP Mon Oct 17 02:57:48 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / dff8e50849 |
| Default Java | Temurin-1.8.0_352-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/5/testReport/ |
| Max. process+thread count | 556 (vs. ulimit of 30000) |
| modules | C: hbase-it U: hbase-it |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/5/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1380238904
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 12s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 4s | master passed |
| +1 :green_heart: | compile | 0m 23s | master passed |
| +1 :green_heart: | shadedjars | 5m 37s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 19s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 10s | the patch passed |
| +1 :green_heart: | compile | 0m 27s | the patch passed |
| +1 :green_heart: | javac | 0m 27s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 22s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 17s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 3s | hbase-it in the patch passed. |
| | | 22m 52s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4954 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux d51cf103e2c1 5.4.0-1088-aws #96~18.04.1-Ubuntu SMP Mon Oct 17 02:57:48 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / c1b9d8ba06 |
| Default Java | Temurin-1.8.0_352-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/3/testReport/ |
| Max. process+thread count | 567 (vs. ulimit of 30000) |
| modules | C: hbase-it U: hbase-it |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/3/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1379182920
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 10s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 49s | master passed |
| +1 :green_heart: | compile | 0m 28s | master passed |
| +1 :green_heart: | checkstyle | 0m 13s | master passed |
| +1 :green_heart: | spotless | 0m 44s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 0m 27s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 26s | the patch passed |
| +1 :green_heart: | compile | 0m 26s | the patch passed |
| +1 :green_heart: | javac | 0m 26s | the patch passed |
| -0 :warning: | checkstyle | 0m 11s | hbase-it: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 8m 50s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. |
| -1 :x: | spotless | 0m 38s | patch has 26 errors when running spotless:check, run spotless:apply to fix. |
| +1 :green_heart: | spotbugs | 0m 32s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. |
| | | 24m 37s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4954 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux 7cd8a92ea71d 5.4.0-131-generic #147-Ubuntu SMP Fri Oct 14 17:07:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / c1b9d8ba06 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-it.txt |
| spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/2/artifact/yetus-general-check/output/patch-spotless.txt |
| Max. process+thread count | 84 (vs. ulimit of 30000) |
| modules | C: hbase-it U: hbase-it |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/2/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1379107373
This code handles the `ratio=1.0` case correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a diff in pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on code in PR #4954:
URL: https://github.com/apache/hbase/pull/4954#discussion_r1068024563
##########
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/monkies/PolicyBasedChaosMonkey.java:
##########
@@ -151,7 +157,9 @@ public boolean isStopped() {
@Override
public void waitForStop() throws InterruptedException {
- monkeyThreadPool.awaitTermination(1, TimeUnit.MINUTES);
+ if (!monkeyThreadPool.awaitTermination(1, TimeUnit.MINUTES)) {
+ LOG.warn("Some pool threads failed to terminate, {}", monkeyThreadPool);
Review Comment:
Strange. `monkeyThreadPool` is not a daemon pool, so if this termination fails for some reason, the process can hang. Let me add a forced shutdown.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1380538429
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 39s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 35s | master passed |
| +1 :green_heart: | compile | 0m 17s | master passed |
| +1 :green_heart: | shadedjars | 4m 11s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 10s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 33s | the patch passed |
| +1 :green_heart: | compile | 0m 17s | the patch passed |
| +1 :green_heart: | javac | 0m 17s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 12s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 11s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 36s | hbase-it in the patch passed. |
| | | 16m 52s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4954 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 17893de185d8 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / dff8e50849 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/5/testReport/ |
| Max. process+thread count | 594 (vs. ulimit of 30000) |
| modules | C: hbase-it U: hbase-it |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/5/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1380482307
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 38s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 58s | master passed |
| +1 :green_heart: | compile | 0m 14s | master passed |
| +1 :green_heart: | shadedjars | 3m 58s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 9s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 3s | the patch passed |
| +1 :green_heart: | compile | 0m 16s | the patch passed |
| +1 :green_heart: | javac | 0m 16s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 1s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 9s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 35s | hbase-it in the patch passed. |
| | | 15m 11s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4954 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 2c34d0b3d4f3 5.4.0-1088-aws #96~18.04.1-Ubuntu SMP Mon Oct 17 02:57:48 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / dff8e50849 |
| Default Java | Temurin-1.8.0_352-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/4/testReport/ |
| Max. process+thread count | 561 (vs. ulimit of 30000) |
| modules | C: hbase-it U: hbase-it |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/4/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1380191681
Good points all around, thanks @Apache9
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1377320545
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 5s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 6s | master passed |
| +1 :green_heart: | compile | 0m 20s | master passed |
| +1 :green_heart: | shadedjars | 4m 23s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 14s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 45s | the patch passed |
| +1 :green_heart: | compile | 0m 19s | the patch passed |
| +1 :green_heart: | javac | 0m 19s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 25s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 12s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 43s | hbase-it in the patch passed. |
| | | 18m 40s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4954 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 0017c29a90f6 5.4.0-131-generic #147-Ubuntu SMP Fri Oct 14 17:07:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 4add5250ed |
| Default Java | Temurin-1.8.0_352-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/1/testReport/ |
| Max. process+thread count | 568 (vs. ulimit of 30000) |
| modules | C: hbase-it U: hbase-it |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/1/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a diff in pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on code in PR #4954:
URL: https://github.com/apache/hbase/pull/4954#discussion_r1068205759
##########
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/monkies/PolicyBasedChaosMonkey.java:
##########
@@ -116,13 +120,31 @@ public static <T> T selectWeightedRandomItem(List<Pair<T, Integer>> items) {
/** Selects and returns ceil(ratio * items.length) random items from the given array */
public static <T> List<T> selectRandomItems(T[] items, float ratio) {
- int selectedNumber = (int) Math.ceil(items.length * ratio);
+ /*
+ * N.b. `ratio` values are not validated. Be aware of excessive values and floating point
+ * arithmetic rounding. Guard against negative input to Random#next() and exceeding boundaries
+ * in call to List#subList.
+ */
- List<T> originalItems = Arrays.asList(items);
- Collections.shuffle(originalItems);
+ // clamp ratio to [0.0,1.0]
+ ratio = Math.max(Math.min(ratio, 1.0f), 0.0f);
- int startIndex = ThreadLocalRandom.current().nextInt(items.length - selectedNumber);
- return originalItems.subList(startIndex, startIndex + selectedNumber);
+ final int selectedNumber = (int) Math.ceil(items.length * ratio);
+
+ // shuffle a copy of the input, not the input.
+ final List<T> shuffledItems = new ArrayList<>(items.length);
+ shuffledItems.addAll(Arrays.asList(items));
+ Collections.shuffle(shuffledItems);
+
+ if (selectedNumber >= items.length) {
+ return shuffledItems;
+ }
+
+ // apply basic sanity check on sublist selection range.
+ final int startIndex =
+ Math.max(0, ThreadLocalRandom.current().nextInt(items.length - selectedNumber));
+ final int endIndex = Math.min(items.length, startIndex + selectedNumber);
+ return shuffledItems.subList(startIndex, endIndex);
Review Comment:
Sure, that's simpler.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1380493487
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 8s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 32s | master passed |
| +1 :green_heart: | compile | 0m 27s | master passed |
| +1 :green_heart: | checkstyle | 0m 10s | master passed |
| +1 :green_heart: | spotless | 0m 40s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 0m 22s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 32s | the patch passed |
| +1 :green_heart: | compile | 0m 26s | the patch passed |
| +1 :green_heart: | javac | 0m 26s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 11s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 8m 44s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. |
| +1 :green_heart: | spotless | 0m 40s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 0m 30s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. |
| | | 24m 23s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/4/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4954 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux 1bbcf7845206 5.4.0-131-generic #147-Ubuntu SMP Fri Oct 14 17:07:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / dff8e50849 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Max. process+thread count | 79 (vs. ulimit of 30000) |
| modules | C: hbase-it U: hbase-it |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/4/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1380489742
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 39s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 3s | master passed |
| +1 :green_heart: | compile | 0m 18s | master passed |
| +1 :green_heart: | shadedjars | 4m 15s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 11s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 37s | the patch passed |
| +1 :green_heart: | compile | 0m 18s | the patch passed |
| +1 :green_heart: | javac | 0m 18s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 20s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 11s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 36s | hbase-it in the patch passed. |
| | | 17m 32s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4954 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 23bfc1180327 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / dff8e50849 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/4/testReport/ |
| Max. process+thread count | 584 (vs. ulimit of 30000) |
| modules | C: hbase-it U: hbase-it |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/4/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1379177988
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 45s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 16s | master passed |
| +1 :green_heart: | compile | 0m 25s | master passed |
| +1 :green_heart: | shadedjars | 5m 24s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 14s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 19s | the patch passed |
| +1 :green_heart: | compile | 0m 21s | the patch passed |
| +1 :green_heart: | javac | 0m 21s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 11s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 15s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 58s | hbase-it in the patch passed. |
| | | 22m 12s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4954 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 855f28b9d7a4 5.4.0-1093-aws #102~18.04.2-Ubuntu SMP Wed Dec 7 00:31:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / c1b9d8ba06 |
| Default Java | Temurin-1.8.0_352-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/2/testReport/ |
| Max. process+thread count | 559 (vs. ulimit of 30000) |
| modules | C: hbase-it U: hbase-it |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/2/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a diff in pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on code in PR #4954:
URL: https://github.com/apache/hbase/pull/4954#discussion_r1067031749
##########
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/monkies/PolicyBasedChaosMonkey.java:
##########
@@ -116,13 +116,13 @@ public static <T> T selectWeightedRandomItem(List<Pair<T, Integer>> items) {
/** Selects and returns ceil(ratio * items.length) random items from the given array */
public static <T> List<T> selectRandomItems(T[] items, float ratio) {
- int selectedNumber = (int) Math.ceil(items.length * ratio);
-
- List<T> originalItems = Arrays.asList(items);
+ final int selectedNumber = (int) Math.ceil(items.length * ratio);
+ final List<T> originalItems = Arrays.asList(items);
Collections.shuffle(originalItems);
-
- int startIndex = ThreadLocalRandom.current().nextInt(items.length - selectedNumber);
- return originalItems.subList(startIndex, startIndex + selectedNumber);
+ final int startIndex =
Review Comment:
Yes, it seems so... and it looks like this patch is not sufficient. Here's the stack trace from branch-2.5.
```
2023-01-10T19:33:35,651 WARN [ChaosMonkey-0] policies.Policy: Exception occurred during performing action: java.lang.IllegalArgumentException: bound must be positive
at java.base/java.util.Random.nextInt(Random.java:322)
at java.base/java.util.concurrent.ThreadLocalRandom.nextInt(ThreadLocalRandom.java:449)
at org.apache.hadoop.hbase.chaos.monkies.PolicyBasedChaosMonkey.selectRandomItems(PolicyBasedChaosMonkey.java:123)
at org.apache.hadoop.hbase.chaos.actions.RollingBatchRestartRsAction.selectServers(RollingBatchRestartRsAction.java:130)
at org.apache.hadoop.hbase.chaos.actions.RollingBatchRestartRsAction.perform(RollingBatchRestartRsAction.java:75)
at org.apache.hadoop.hbase.chaos.policies.DoActionsOncePolicy.runOneIteration(DoActionsOncePolicy.java:48)
at org.apache.hadoop.hbase.chaos.policies.PeriodicPolicy.run(PeriodicPolicy.java:41)
at org.apache.hadoop.hbase.chaos.policies.CompositeSequentialPolicy.run(CompositeSequentialPolicy.java:42)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1379164264
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 40s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 56s | master passed |
| +1 :green_heart: | compile | 0m 17s | master passed |
| +1 :green_heart: | shadedjars | 4m 14s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 13s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 2m 38s | the patch passed |
| +1 :green_heart: | compile | 0m 18s | the patch passed |
| +1 :green_heart: | javac | 0m 18s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 13s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 10s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 36s | hbase-it in the patch passed. |
| | | 17m 6s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4954 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 41bd126d4495 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / c1b9d8ba06 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/2/testReport/ |
| Max. process+thread count | 580 (vs. ulimit of 30000) |
| modules | C: hbase-it U: hbase-it |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/2/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1380247475
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 22s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 42s | master passed |
| +1 :green_heart: | compile | 0m 48s | master passed |
| +1 :green_heart: | checkstyle | 0m 16s | master passed |
| +1 :green_heart: | spotless | 1m 10s | branch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 0m 36s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 59s | the patch passed |
| +1 :green_heart: | compile | 0m 47s | the patch passed |
| +1 :green_heart: | javac | 0m 47s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 17s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 10m 16s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. |
| +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. |
| +1 :green_heart: | spotbugs | 0m 32s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 9s | The patch does not generate ASF License warnings. |
| | | 30m 19s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4954 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile |
| uname | Linux 850f83e9e6ab 5.4.0-131-generic #147-Ubuntu SMP Fri Oct 14 17:07:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / c1b9d8ba06 |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Max. process+thread count | 76 (vs. ulimit of 30000) |
| modules | C: hbase-it U: hbase-it |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/3/console |
| versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1379094607
We have `1.0f` as default ratio values for a couple constants in `MonkeyConstants.java`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] ndimiduk merged pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
ndimiduk merged PR #4954:
URL: https://github.com/apache/hbase/pull/4954
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4954: HBASE-27563 ChaosMonkey sometimes generates invalid boundaries for random item selection
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #4954:
URL: https://github.com/apache/hbase/pull/4954#issuecomment-1377321579
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 19s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 37s | master passed |
| +1 :green_heart: | compile | 0m 23s | master passed |
| +1 :green_heart: | shadedjars | 4m 47s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 15s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 8s | the patch passed |
| +1 :green_heart: | compile | 0m 21s | the patch passed |
| +1 :green_heart: | javac | 0m 21s | the patch passed |
| +1 :green_heart: | shadedjars | 4m 29s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 15s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 58s | hbase-it in the patch passed. |
| | | 20m 12s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/4954 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 7f1c36041360 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 4add5250ed |
| Default Java | Eclipse Adoptium-11.0.17+8 |
| Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/1/testReport/ |
| Max. process+thread count | 590 (vs. ulimit of 30000) |
| modules | C: hbase-it U: hbase-it |
| Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4954/1/console |
| versions | git=2.34.1 maven=3.8.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org