You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2020/03/02 19:25:51 UTC
[GitHub] [hadoop] karthikhw opened a new pull request #1870: HDFS-15201
SnapshotCounter hits MaxSnapshotID limit
karthikhw opened a new pull request #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] szetszwo commented on issue #1870: HDFS-15201
SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
szetszwo commented on issue #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#issuecomment-599286169
+1 the latest change looks good. Thanks @karthikhw
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] szetszwo commented on issue #1870: HDFS-15201
SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
szetszwo commented on issue #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#issuecomment-597141984
The changes look good. Just a question: why changing getMaxSnapshotID() from -1 to -2?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1870: HDFS-15201
SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on issue #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#issuecomment-594951319
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 31s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 19m 2s | trunk passed |
| +1 :green_heart: | compile | 1m 6s | trunk passed |
| +1 :green_heart: | checkstyle | 0m 46s | trunk passed |
| +1 :green_heart: | mvnsite | 1m 12s | trunk passed |
| +1 :green_heart: | shadedclient | 15m 58s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 44s | trunk passed |
| +0 :ok: | spotbugs | 2m 51s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 49s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 6s | the patch passed |
| +1 :green_heart: | compile | 1m 3s | the patch passed |
| +1 :green_heart: | javac | 1m 3s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 39s | hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 10 unchanged - 1 fixed = 10 total (was 11) |
| +1 :green_heart: | mvnsite | 1m 10s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 13m 52s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 40s | the patch passed |
| +1 :green_heart: | findbugs | 2m 53s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 92m 45s | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 40s | The patch does not generate ASF License warnings. |
| | | 158m 10s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.TestEncryptionZonesWithKMS |
| | hadoop.hdfs.TestEncryptionZones |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1870/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/1870 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux bb84c0e515d8 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 3afd4cb |
| Default Java | 1.8.0_242 |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1870/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1870/3/testReport/ |
| Max. process+thread count | 4515 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1870/3/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] karthikhw commented on a change in pull request #1870:
HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
karthikhw commented on a change in pull request #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#discussion_r387906221
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/SnapshotManager.java
##########
@@ -541,8 +541,8 @@ public void clearSnapshottableDirs() {
*
* @return maximum allowable snapshot ID.
*/
- public int getMaxSnapshotID() {
- return ((1 << SNAPSHOT_ID_BIT_WIDTH) - 1);
+ public int getMaxSnapshotID() {
+ return (1 << SNAPSHOT_ID_BIT_WIDTH);
Review comment:
Thank you @mukul1987. I didn't understand this "as we would only have one snapshot id here".
SNAPSHOT_ID_BIT_WIDTH has no relation with anyone. Not sure why would we need index -1.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on issue #1870: HDFS-15201
SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
jojochuang commented on issue #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#issuecomment-593718653
@karthikhw can you double check the failed tests? especially the snapshot tests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] karthikhw commented on issue #1870: HDFS-15201
SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
karthikhw commented on issue #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#issuecomment-598484891
Thank you @szetszwo I changed back to -1.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] karthikhw commented on a change in pull request #1870:
HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
karthikhw commented on a change in pull request #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#discussion_r386677308
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/SnapshotManager.java
##########
@@ -96,7 +96,7 @@
private final boolean snapshotDiffAllowSnapRootDescendant;
private final AtomicInteger numSnapshots = new AtomicInteger();
- private static final int SNAPSHOT_ID_BIT_WIDTH = 24;
+ private static final int SNAPSHOT_ID_BIT_WIDTH = 31;
Review comment:
Sure @arp7 I will set to 28 then.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] karthikhw commented on issue #1870: HDFS-15201
SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
karthikhw commented on issue #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#issuecomment-597278362
@szetszwo
Changed getMaxSnapshotID() from -1 to -2 because of CURRENT_STATE_ID(Integer.MAX_VALUE - 1) that have -1 already.
If SNAPSHOT_ID_BIT_WIDTH is 28 then we are ok with -1 but later change in SNAPSHOT_ID_BIT_WIDTH to 31 then user have to update getMaxSnapshotID() from -1 to -2.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] karthikhw commented on a change in pull request #1870:
HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
karthikhw commented on a change in pull request #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#discussion_r387953842
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotManager.java
##########
@@ -92,4 +92,15 @@ public void testSnapshotLimits() throws Exception {
StringUtils.toLowerCase(se.getMessage()).contains("rollover"));
}
}
+
+
+ @Test
+ public void testValidateSnapshotIDWidth() {
+ FSDirectory fsdir = mock(FSDirectory.class);
+ SnapshotManager snapshotManager = new SnapshotManager(new Configuration(),
+ fsdir);
+ Assert.assertTrue(snapshotManager.
Review comment:
Added!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] arp7 commented on a change in pull request #1870:
HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
arp7 commented on a change in pull request #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#discussion_r386626461
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/SnapshotManager.java
##########
@@ -96,7 +96,7 @@
private final boolean snapshotDiffAllowSnapRootDescendant;
private final AtomicInteger numSnapshots = new AtomicInteger();
- private static final int SNAPSHOT_ID_BIT_WIDTH = 24;
+ private static final int SNAPSHOT_ID_BIT_WIDTH = 31;
Review comment:
I believe @szetszwo recommended setting this to 28 bits for now (something less than 31).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] karthikhw commented on issue #1870: HDFS-15201
SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
karthikhw commented on issue #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#issuecomment-596697248
@arp7 @jojochuang Can you please review this change if you get sometime?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] karthikhw commented on issue #1870: HDFS-15201
SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
karthikhw commented on issue #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#issuecomment-593652779
Submitted new PR with the changes requested.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1870: HDFS-15201
SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on issue #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#issuecomment-593714119
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 31s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 18m 53s | trunk passed |
| +1 :green_heart: | compile | 1m 9s | trunk passed |
| +1 :green_heart: | checkstyle | 0m 44s | trunk passed |
| +1 :green_heart: | mvnsite | 1m 18s | trunk passed |
| +1 :green_heart: | shadedclient | 16m 3s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 46s | trunk passed |
| +0 :ok: | spotbugs | 2m 49s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 48s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 7s | the patch passed |
| +1 :green_heart: | compile | 1m 0s | the patch passed |
| +1 :green_heart: | javac | 1m 0s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 38s | hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 10 unchanged - 1 fixed = 10 total (was 11) |
| +1 :green_heart: | mvnsite | 1m 7s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 39s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 44s | the patch passed |
| +1 :green_heart: | findbugs | 2m 54s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 92m 36s | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 41s | The patch does not generate ASF License warnings. |
| | | 159m 57s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.TestEncryptionZonesWithKMS |
| | hadoop.hdfs.TestEncryptionZones |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1870/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/1870 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 828dfedc29e0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / edc2e9d |
| Default Java | 1.8.0_242 |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1870/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1870/2/testReport/ |
| Max. process+thread count | 4480 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1870/2/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] lokeshj1703 commented on issue #1870: HDFS-15201
SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
lokeshj1703 commented on issue #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#issuecomment-603136710
@karthikhw Thanks for the contribution! @jojochuang @mukul1987 @arp7 @szetszwo Thanks for reviewing the PR! I have committed it to master branch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1870: HDFS-15201
SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on issue #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#issuecomment-593644106
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 19m 26s | trunk passed |
| +1 :green_heart: | compile | 1m 8s | trunk passed |
| +1 :green_heart: | checkstyle | 0m 45s | trunk passed |
| +1 :green_heart: | mvnsite | 1m 15s | trunk passed |
| +1 :green_heart: | shadedclient | 16m 3s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 47s | trunk passed |
| +0 :ok: | spotbugs | 2m 50s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 48s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 7s | the patch passed |
| +1 :green_heart: | compile | 1m 1s | the patch passed |
| +1 :green_heart: | javac | 1m 1s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 37s | hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 10 unchanged - 1 fixed = 10 total (was 11) |
| +1 :green_heart: | mvnsite | 1m 8s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 13m 58s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 42s | the patch passed |
| +1 :green_heart: | findbugs | 2m 58s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 80m 30s | hadoop-hdfs in the patch passed. |
| +0 :ok: | asflicense | 0m 40s | ASF License check generated no output? |
| | | 146m 50s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.namenode.snapshot.TestRandomOpsWithSnapshots |
| | hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot |
| | hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens |
| | hadoop.hdfs.server.namenode.TestCacheDirectives |
| | hadoop.hdfs.server.namenode.snapshot.TestSnapshot |
| | hadoop.hdfs.server.namenode.TestFsck |
| | hadoop.hdfs.server.namenode.TestLargeDirectoryDelete |
| | hadoop.hdfs.server.namenode.TestReencryptionWithKMS |
| | hadoop.hdfs.server.namenode.TestNestedEncryptionZones |
| | hadoop.hdfs.server.namenode.TestFSImage |
| | hadoop.hdfs.server.namenode.TestReencryption |
| | hadoop.hdfs.server.namenode.sps.TestStoragePolicySatisfierWithStripedFile |
| | hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd |
| | hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
| | hadoop.hdfs.server.namenode.TestAuditLoggerWithCommands |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
| | hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys |
| | hadoop.hdfs.server.namenode.snapshot.TestSnapshotStatsMXBean |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1870/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/1870 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 45d4fd00ddea 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / edc2e9d |
| Default Java | 1.8.0_242 |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1870/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1870/1/testReport/ |
| Max. process+thread count | 4152 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1870/1/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] szetszwo commented on issue #1870: HDFS-15201
SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
szetszwo commented on issue #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#issuecomment-598480242
Let's keep it "-1" since we are using 28 for the moment. If there still a problem later on, we should think about what to do. We do not necessarily change it to 31 at that time.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on issue #1870: HDFS-15201
SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
jojochuang commented on issue #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#issuecomment-593719088
I've gone through all the usage of the snapshot id. The only concern i had was bitwise operations on the snapshot id, but i didn't find any. Widening the allowed range shouldn't be a problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] mukul1987 commented on a change in pull request #1870:
HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
mukul1987 commented on a change in pull request #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#discussion_r386768106
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/SnapshotManager.java
##########
@@ -541,8 +541,8 @@ public void clearSnapshottableDirs() {
*
* @return maximum allowable snapshot ID.
*/
- public int getMaxSnapshotID() {
- return ((1 << SNAPSHOT_ID_BIT_WIDTH) - 1);
+ public int getMaxSnapshotID() {
+ return (1 << SNAPSHOT_ID_BIT_WIDTH);
Review comment:
I think we can still leave this as max index -1, as we would only have one snapshot id here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on a change in pull request #1870:
HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
jojochuang commented on a change in pull request #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#discussion_r386750242
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotManager.java
##########
@@ -92,4 +92,15 @@ public void testSnapshotLimits() throws Exception {
StringUtils.toLowerCase(se.getMessage()).contains("rollover"));
}
}
+
+
+ @Test
+ public void testValidateSnapshotIDWidth() {
+ FSDirectory fsdir = mock(FSDirectory.class);
+ SnapshotManager snapshotManager = new SnapshotManager(new Configuration(),
+ fsdir);
+ Assert.assertTrue(snapshotManager.
Review comment:
can you add a description/comment to explain this assertion.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] lokeshj1703 merged pull request #1870: HDFS-15201
SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
lokeshj1703 merged pull request #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] karthikhw commented on issue #1870: HDFS-15201
SnapshotCounter hits MaxSnapshotID limit
Posted by GitBox <gi...@apache.org>.
karthikhw commented on issue #1870: HDFS-15201 SnapshotCounter hits MaxSnapshotID limit
URL: https://github.com/apache/hadoop/pull/1870#issuecomment-594957888
It looks test case failure is un-related to this issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org