You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2021/11/16 22:51:25 UTC
[GitHub] [hbase] joswiatek opened a new pull request #3852: HBASE-22458 Add UNSET_SNAPSHOT_PROP and fix TTL defaulting
joswiatek opened a new pull request #3852:
URL: https://github.com/apache/hbase/pull/3852
See JIRA for detail: https://issues.apache.org/jira/browse/HBASE-26458
The bug is that the default value for TTL when no explicit TTL is -1, but the snapshot validation that applies `hbase.master.snapshot.ttl` looks for `0`. Both should use -1, and share the value in a constant.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #3852: HBASE-22458 Add UNSET_SNAPSHOT_PROP and fix TTL defaulting
Posted by GitBox <gi...@apache.org>.
virajjasani commented on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-971428634
The only case that might require behaviour change is when user specifies TTL 0:
```
> snapshot 'mytable', 'snapshot1234', {TTL => 0}
```
This snapshot would also be eligible to be deleted as per config `hbase.master.snapshot.ttl` because this config is applicable at entire cluster.
If use specifies TTL value less than -1, then that snapshot will never be deleted based on TTL.
Here is the current behaviour:
1. If client wants to create specific snapshot that should live forever regardless of `hbase.master.snapshot.ttl` config value, then they should specify TTL < -1.
2. If we don't want to enable TTL at entire cluster level, no need of any config change because the default value of `hbase.master.snapshot.ttl` is 0, which represents TTL Forever at server side.
It's also important to note that SnapshotCleanerChore can be enabled/disabled using shell command: `snapshot_cleanup_switch`. More details in the section `Enable/Disable Snapshot Auto Cleanup on running cluster` [here](http://hbase.apache.org/book.html#ops.snapshots.takeasnapshot)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #3852: HBASE-26458 Update Snapshot TTL doc
Posted by GitBox <gi...@apache.org>.
Apache9 commented on a change in pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#discussion_r753632040
##########
File path: src/main/asciidoc/_chapters/ops_mgt.adoc
##########
@@ -3150,23 +3150,24 @@ TTL some notion of optional TTL (and optional default TTL) for snapshots could b
hbase> snapshot 'mytable', 'snapshot1234', {TTL => 86400}
----
-The above command creates snapshot `snapshot1234` with TTL of 86400 sec(24 hours)
+The above command creates snapshot `snapshot1234` with TTL of 86400 sec (24 hours)
and hence, the snapshot is supposed to be cleaned up after 24 hours
.Default Snapshot TTL:
-
-- FOREVER by default
-- User specified Default TTL with config `hbase.master.snapshot.ttl`
-
-
-While creating a Snapshot, if TTL in seconds is not specified, by default the snapshot
-would not be deleted automatically. i.e. it would be retained forever until it is
-manually deleted. However, the user can update this default TTL behavior by
-providing default TTL in sec for key: `hbase.master.snapshot.ttl`.
-Value 0 for this config indicates TTL: FOREVER
-
+- User specified default TTL with config `hbase.master.snapshot.ttl`
+- FOREVER if `hbase.master.snapshot.ttl` is not set
+
+While creating a snapshot, if TTL in seconds is not explicitly specified, the above logic will be
+followed to determine the TTL. If no configs are changed, the default behavior is that all snapshots
+will be retained forever (until manual deletion). If a different default TTL behavior is desired,
+`hbase.master.snapshot.ttl` can be set to a default TTL in seconds. Any snapshot created without
+an explicit TTL will take this new value.
+
+NOTE: If `hbase.master.snapshot.ttl` is set, a snapshot with an explicit {TTL => 0} or
+{TTL => -1} will also take this value. In this case, a TTL < -1 (such as {TTL => -2} should be used
+to indicate FOREVER.
Review comment:
The behavior is a bit strange, is this by design? Or it just works like this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani merged pull request #3852: HBASE-26458 Update Snapshot TTL doc
Posted by GitBox <gi...@apache.org>.
virajjasani merged pull request #3852:
URL: https://github.com/apache/hbase/pull/3852
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3852: HBASE-26458 Update Snapshot TTL doc
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-973211636
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 58s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 4m 29s | master passed |
| +1 :green_heart: | compile | 9m 54s | master passed |
| +1 :green_heart: | checkstyle | 2m 12s | master passed |
| +0 :ok: | refguide | 3m 47s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. |
| +1 :green_heart: | spotbugs | 15m 28s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 35s | the patch passed |
| +1 :green_heart: | compile | 10m 21s | the patch passed |
| +1 :green_heart: | javac | 10m 21s | the patch passed |
| +1 :green_heart: | checkstyle | 2m 12s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +0 :ok: | refguide | 4m 5s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. |
| +1 :green_heart: | hadoopcheck | 20m 49s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. |
| +1 :green_heart: | spotbugs | 14m 41s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 56s | The patch does not generate ASF License warnings. |
| | | 104m 40s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3852 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile refguide |
| uname | Linux 9d17c07c44f8 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / b2571df7ae |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| refguide | https://nightlies.apache.org/hbase/HBase/HBase-PreCommit-GitHub-PR/PR-3852/2/yetus-general-check/output/branch-site/book.html |
| refguide | https://nightlies.apache.org/hbase/HBase/HBase-PreCommit-GitHub-PR/PR-3852/2/yetus-general-check/output/patch-site/book.html |
| Max. process+thread count | 141 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server . U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/2/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #3852: HBASE-26458 Update Snapshot TTL doc
Posted by GitBox <gi...@apache.org>.
virajjasani commented on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-976398282
Thanks @joswiatek. Left one comment on #3857 for branch-1 backport. Once done, will merge both PRs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3852: HBASE-22458 Add UNSET_SNAPSHOT_PROP and fix TTL defaulting
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-971132203
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 8s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 4m 38s | master passed |
| +1 :green_heart: | compile | 1m 57s | master passed |
| +1 :green_heart: | shadedjars | 9m 4s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 24s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 16s | the patch passed |
| +1 :green_heart: | compile | 1m 57s | the patch passed |
| +1 :green_heart: | javac | 1m 57s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 4s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 20s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 56s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 35s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 213m 2s | hbase-server in the patch passed. |
| | | 254m 17s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3852 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 021f437fa253 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 1c47f80d83 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/1/testReport/ |
| Max. process+thread count | 3918 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #3852: HBASE-26458 Update Snapshot TTL doc
Posted by GitBox <gi...@apache.org>.
virajjasani commented on a change in pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#discussion_r753019998
##########
File path: src/main/asciidoc/_chapters/ops_mgt.adoc
##########
@@ -3150,23 +3150,24 @@ TTL some notion of optional TTL (and optional default TTL) for snapshots could b
hbase> snapshot 'mytable', 'snapshot1234', {TTL => 86400}
----
-The above command creates snapshot `snapshot1234` with TTL of 86400 sec(24 hours)
+The above command creates snapshot `snapshot1234` with TTL of 86400 sec (24 hours)
and hence, the snapshot is supposed to be cleaned up after 24 hours
.Default Snapshot TTL:
-
-- FOREVER by default
-- User specified Default TTL with config `hbase.master.snapshot.ttl`
-
-
-While creating a Snapshot, if TTL in seconds is not specified, by default the snapshot
-would not be deleted automatically. i.e. it would be retained forever until it is
-manually deleted. However, the user can update this default TTL behavior by
-providing default TTL in sec for key: `hbase.master.snapshot.ttl`.
-Value 0 for this config indicates TTL: FOREVER
-
+- User specified default TTL with config `hbase.master.snapshot.ttl`
+- FOREVER if `hbase.master.snapshot.ttl` is not set
+
+While creating a snapshot, if TTL in seconds is not explicitly specified, the above logic will be
+followed to determine the TTL. If no configs are changed, the default behavior is that all snapshots
+will be retained forever (until manual deletion). If a different default TTL behavior is desired,
+`hbase.master.snapshot.ttl` can be set to a default TTL in seconds. Any snapshot created without
+an explicit TTL will take this new value.
+
+NOTE: If `hbase.master.snapshot.ttl` is set, a snapshot with an explicit {TTL => 0} or
+{TTL => -1} will also take this value. In this case, a TTL < -1 (such as {TTL => -2} should be used
+to indicate FOREVER.
Review comment:
Thanks @joswiatek.
Looks good. After this line, shall we sum up with these points? Or maybe it's only me thinking that these summary points would get more attention than description and notes above?
```
To sum-up,
1. Snapshot with TTL value < -1 will stay forever regardless of any server side config changes (until deleted manually by user).
2. Snapshot with TTL value > 0 will be deleted soon after TTL expires.
3. Snapshot created without specifying TTL will always have TTL value represented by config `hbase.master.snapshot.ttl`. Default value of this config is 0, which represents: keep the snapshot forever (until deleted manually by user).
4. From client side, TTL value 0 or -1 should not be provided because they will be treated same as snapshot without TTL (same as above point 3) and hence will use TTL as per value represented by config `hbase.master.snapshot.ttl`.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] joswiatek commented on pull request #3852: HBASE-26458 Update Snapshot TTL doc
Posted by GitBox <gi...@apache.org>.
joswiatek commented on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-975977333
I've included the additional doc suggestions, thanks for the review y'all
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3852: HBASE-22458 Add UNSET_SNAPSHOT_PROP and fix TTL defaulting
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-971072115
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 28s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 4m 42s | master passed |
| +1 :green_heart: | compile | 2m 13s | master passed |
| +1 :green_heart: | shadedjars | 8m 17s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 35s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 32s | the patch passed |
| +1 :green_heart: | compile | 2m 12s | the patch passed |
| +1 :green_heart: | javac | 2m 12s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 20s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 34s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 2s | hbase-common in the patch passed. |
| +1 :green_heart: | unit | 1m 32s | hbase-client in the patch passed. |
| +1 :green_heart: | unit | 137m 33s | hbase-server in the patch passed. |
| | | 178m 29s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3852 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux de2c7bf6c6fa 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 1c47f80d83 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/1/testReport/ |
| Max. process+thread count | 4247 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] joswiatek commented on pull request #3852: HBASE-26458 Add UNSET_SNAPSHOT_PROP and fix TTL defaulting
Posted by GitBox <gi...@apache.org>.
joswiatek commented on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-972199303
Thank you for digging into this! I totally missed the protobuf component. It is great to see this is working on master HEAD.
After reviewing these new protobuf details, I think the behavior where `hbase.master.snapshot.ttl` is ignored is only present on branch-1. On branch-1, the builder just directly takes the value from the snapshotProps, so -1 is actually sent and it does not match `NO_SNAPSHOT_TTL_SPECIFIED`, and the config is not applied. See [here](https://github.com/apache/hbase/blob/0d214ff53e3b8cb3bccbc01815ccbc9a03dfabb4/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java#L3711-L3717).
```
SnapshotDescription.Builder builder = SnapshotDescription.newBuilder();
builder.setTable(tableName.getNameAsString());
builder.setName(snapshotName);
builder.setType(type);
builder.setTtl(getTtlFromSnapshotProps(snapshotProps));
snapshot(builder.build());
```
On a branch-1 build, I do see snapshots being created with a -1 TTL when the default config is set.
#### Configs:
```
<property>
<name>hbase.master.snapshot.ttl</name>
<value>600</value>
</property>
```
#### Shell commands and outputs
```
snapshot 't1', 's1'
```
Snapshot is actually created with a TTL of `-1` instead of 600
> updating default value in proto message is considered as incompatible change
I am not experienced with protobuf, so I will take your word about defaults here. One question: is changing the logic around building the protobuf also an incompatible change? If we can update the TTL conversion logic to allow for sending `-1`, then the code changes in this PR will make sense and allow for the currently documented behavior to be fulfilled.
If that's not possible, I can update this PR with the clarifying doc changes as suggested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3852: HBASE-26458 Update Snapshot TTL doc
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-976232207
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 5s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 5m 17s | master passed |
| +1 :green_heart: | compile | 3m 33s | master passed |
| +1 :green_heart: | shadedjars | 10m 27s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 4m 33s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 5m 1s | the patch passed |
| +1 :green_heart: | compile | 3m 31s | the patch passed |
| +1 :green_heart: | javac | 3m 31s | the patch passed |
| +1 :green_heart: | shadedjars | 10m 29s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 4m 36s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 505m 5s | root in the patch passed. |
| | | 556m 51s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3852 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 39cbe1f7cbd2 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 04635f405a |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/3/testReport/ |
| Max. process+thread count | 4002 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server . U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/3/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3852: HBASE-22458 Add UNSET_SNAPSHOT_PROP and fix TTL defaulting
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-970870606
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 28s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 4m 1s | master passed |
| +1 :green_heart: | compile | 5m 1s | master passed |
| +1 :green_heart: | checkstyle | 1m 58s | master passed |
| +1 :green_heart: | spotbugs | 3m 51s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 47s | the patch passed |
| +1 :green_heart: | compile | 4m 52s | the patch passed |
| +1 :green_heart: | javac | 4m 52s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 54s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 19m 26s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. |
| +1 :green_heart: | spotbugs | 4m 19s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 41s | The patch does not generate ASF License warnings. |
| | | 59m 55s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3852 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux b6a16d707873 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 1c47f80d83 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Max. process+thread count | 96 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #3852: HBASE-26458 Add UNSET_SNAPSHOT_PROP and fix TTL defaulting
Posted by GitBox <gi...@apache.org>.
virajjasani commented on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-971465945
I just tested this locally on master branch.
**Configs:**
<property>
<name>hbase.master.cleaner.snapshot.interval</name>
<value>10000</value>
</property>
**Shell commands and respective outputs:**
> snapshot 't1', 's2'
Took 2.1034 seconds
> snapshot 't1', 's3', {TTL => -2}
Took 1.0925 seconds
=> [{"TTL"=>-2}]
> snapshot 't1', 's1', {TTL => 5}
Took 2.1023 seconds
=> [{"TTL"=>5}]
<img width="1630" alt="Screenshot 2021-11-17 at 4 17 11 PM" src="https://user-images.githubusercontent.com/34790606/142187725-e0f67af3-90bc-47ef-bd28-f73285c27dd3.png">
After 5sec, the snapshot was gone.
<img width="1568" alt="Screenshot 2021-11-17 at 4 19 32 PM" src="https://user-images.githubusercontent.com/34790606/142187833-749fb7a2-5a31-4956-9f65-b7fb8bda36ee.png">
**Configs:**
<property>
<name>hbase.master.cleaner.snapshot.interval</name>
<value>10000</value>
</property>
<property>
<name>hbase.master.snapshot.ttl</name>
<value>20</value>
</property>
**Shell commands and respective outputs:**
> snapshot 't1', 's4'
Took 2.7291 seconds
Snapshot 's4' gets created with TTL 20 sec because of config `hbase.master.snapshot.ttl`:
<img width="1584" alt="Screenshot 2021-11-17 at 4 21 49 PM" src="https://user-images.githubusercontent.com/34790606/142188010-15a2abf1-26aa-4553-a803-ec2e96b5803d.png">
After 20s, it was gone:
<img width="1539" alt="Screenshot 2021-11-17 at 4 22 04 PM" src="https://user-images.githubusercontent.com/34790606/142188219-1ada2b96-0a64-4f6c-b5f2-711b3abfc8e8.png">
> snapshot 't1', 's5', {TTL => -10}
Took 2.1079 seconds
=> [{"TTL"=>-10}]
<img width="1618" alt="Screenshot 2021-11-17 at 4 23 43 PM" src="https://user-images.githubusercontent.com/34790606/142188278-ed70511c-bea6-44f8-a3d6-c70c674bbd57.png">
1. TTL with negative values will stay forever regardless of any server side config changes.
2. TTL with positive values will be gone after TTL expires.
3. Snapshot created without specifying TTL will always have TTL values represented by config `hbase.master.snapshot.ttl`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani edited a comment on pull request #3852: HBASE-26458 Add UNSET_SNAPSHOT_PROP and fix TTL defaulting
Posted by GitBox <gi...@apache.org>.
virajjasani edited a comment on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-971465945
I just tested on the latest HEAD of master branch.
**Configs:**
```
<property>
<name>hbase.master.cleaner.snapshot.interval</name>
<value>10000</value>
</property>
```
**Shell commands and respective outputs:**
> snapshot 't1', 's2'
Took 2.1034 seconds
> snapshot 't1', 's3', {TTL => -2}
Took 1.0925 seconds
=> [{"TTL"=>-2}]
> snapshot 't1', 's1', {TTL => 5}
Took 2.1023 seconds
=> [{"TTL"=>5}]
<img width="1630" alt="Screenshot 2021-11-17 at 4 17 11 PM" src="https://user-images.githubusercontent.com/34790606/142187725-e0f67af3-90bc-47ef-bd28-f73285c27dd3.png">
After 5sec, the snapshot was gone.
<img width="1568" alt="Screenshot 2021-11-17 at 4 19 32 PM" src="https://user-images.githubusercontent.com/34790606/142187833-749fb7a2-5a31-4956-9f65-b7fb8bda36ee.png">
**Configs:**
```
<property>
<name>hbase.master.cleaner.snapshot.interval</name>
<value>10000</value>
</property>
<property>
<name>hbase.master.snapshot.ttl</name>
<value>20</value>
</property>
```
**Shell commands and respective outputs:**
> snapshot 't1', 's4'
Took 2.7291 seconds
Snapshot 's4' gets created with TTL 20 sec because of config `hbase.master.snapshot.ttl`:
<img width="1584" alt="Screenshot 2021-11-17 at 4 21 49 PM" src="https://user-images.githubusercontent.com/34790606/142188010-15a2abf1-26aa-4553-a803-ec2e96b5803d.png">
After 20s, it was gone:
<img width="1539" alt="Screenshot 2021-11-17 at 4 22 04 PM" src="https://user-images.githubusercontent.com/34790606/142188219-1ada2b96-0a64-4f6c-b5f2-711b3abfc8e8.png">
> snapshot 't1', 's5', {TTL => -10}
Took 2.1079 seconds
=> [{"TTL"=>-10}]
<img width="1618" alt="Screenshot 2021-11-17 at 4 23 43 PM" src="https://user-images.githubusercontent.com/34790606/142188278-ed70511c-bea6-44f8-a3d6-c70c674bbd57.png">
1. TTL with negative values will stay forever regardless of any server side config changes.
2. TTL with positive values will be gone after TTL expires.
3. Snapshot created without specifying TTL will always have TTL values represented by config `hbase.master.snapshot.ttl`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3852: HBASE-26458 Update Snapshot TTL doc
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-973661579
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 6m 14s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 4m 19s | master passed |
| +1 :green_heart: | compile | 3m 0s | master passed |
| +1 :green_heart: | shadedjars | 9m 8s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 3m 33s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 15s | the patch passed |
| +1 :green_heart: | compile | 3m 0s | the patch passed |
| +1 :green_heart: | javac | 3m 0s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 9s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 3m 38s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 425m 50s | root in the patch passed. |
| | | 475m 42s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3852 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 8d50511c12e6 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / b2571df7ae |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/2/testReport/ |
| Max. process+thread count | 3927 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server . U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #3852: HBASE-26458 Update Snapshot TTL doc
Posted by GitBox <gi...@apache.org>.
virajjasani commented on a change in pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#discussion_r753019998
##########
File path: src/main/asciidoc/_chapters/ops_mgt.adoc
##########
@@ -3150,23 +3150,24 @@ TTL some notion of optional TTL (and optional default TTL) for snapshots could b
hbase> snapshot 'mytable', 'snapshot1234', {TTL => 86400}
----
-The above command creates snapshot `snapshot1234` with TTL of 86400 sec(24 hours)
+The above command creates snapshot `snapshot1234` with TTL of 86400 sec (24 hours)
and hence, the snapshot is supposed to be cleaned up after 24 hours
.Default Snapshot TTL:
-
-- FOREVER by default
-- User specified Default TTL with config `hbase.master.snapshot.ttl`
-
-
-While creating a Snapshot, if TTL in seconds is not specified, by default the snapshot
-would not be deleted automatically. i.e. it would be retained forever until it is
-manually deleted. However, the user can update this default TTL behavior by
-providing default TTL in sec for key: `hbase.master.snapshot.ttl`.
-Value 0 for this config indicates TTL: FOREVER
-
+- User specified default TTL with config `hbase.master.snapshot.ttl`
+- FOREVER if `hbase.master.snapshot.ttl` is not set
+
+While creating a snapshot, if TTL in seconds is not explicitly specified, the above logic will be
+followed to determine the TTL. If no configs are changed, the default behavior is that all snapshots
+will be retained forever (until manual deletion). If a different default TTL behavior is desired,
+`hbase.master.snapshot.ttl` can be set to a default TTL in seconds. Any snapshot created without
+an explicit TTL will take this new value.
+
+NOTE: If `hbase.master.snapshot.ttl` is set, a snapshot with an explicit {TTL => 0} or
+{TTL => -1} will also take this value. In this case, a TTL < -1 (such as {TTL => -2} should be used
+to indicate FOREVER.
Review comment:
Thanks @joswiatek.
Looks good. After this line, shall we sum up with these points?
```
To sum-up,
1. Snapshot with TTL value < -1 will stay forever regardless of any server side config changes (until deleted manually by user).
2. Snapshot with TTL value > 0 will be deleted soon after TTL expires.
3. Snapshot created without specifying TTL will always have TTL value represented by config `hbase.master.snapshot.ttl`. Default value of this config is 0, which represents: keep the snapshot forever (until deleted manually by user).
4. From client side, TTL value 0 or -1 should not be provided because they will be treated same as snapshot without TTL (same as above point 3) and hence will use TTL as per value represented by config `hbase.master.snapshot.ttl`.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #3852: HBASE-26458 Update Snapshot TTL doc
Posted by GitBox <gi...@apache.org>.
virajjasani commented on a change in pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#discussion_r753652443
##########
File path: src/main/asciidoc/_chapters/ops_mgt.adoc
##########
@@ -3150,23 +3150,24 @@ TTL some notion of optional TTL (and optional default TTL) for snapshots could b
hbase> snapshot 'mytable', 'snapshot1234', {TTL => 86400}
----
-The above command creates snapshot `snapshot1234` with TTL of 86400 sec(24 hours)
+The above command creates snapshot `snapshot1234` with TTL of 86400 sec (24 hours)
and hence, the snapshot is supposed to be cleaned up after 24 hours
.Default Snapshot TTL:
-
-- FOREVER by default
-- User specified Default TTL with config `hbase.master.snapshot.ttl`
-
-
-While creating a Snapshot, if TTL in seconds is not specified, by default the snapshot
-would not be deleted automatically. i.e. it would be retained forever until it is
-manually deleted. However, the user can update this default TTL behavior by
-providing default TTL in sec for key: `hbase.master.snapshot.ttl`.
-Value 0 for this config indicates TTL: FOREVER
-
+- User specified default TTL with config `hbase.master.snapshot.ttl`
+- FOREVER if `hbase.master.snapshot.ttl` is not set
+
+While creating a snapshot, if TTL in seconds is not explicitly specified, the above logic will be
+followed to determine the TTL. If no configs are changed, the default behavior is that all snapshots
+will be retained forever (until manual deletion). If a different default TTL behavior is desired,
+`hbase.master.snapshot.ttl` can be set to a default TTL in seconds. Any snapshot created without
+an explicit TTL will take this new value.
+
+NOTE: If `hbase.master.snapshot.ttl` is set, a snapshot with an explicit {TTL => 0} or
+{TTL => -1} will also take this value. In this case, a TTL < -1 (such as {TTL => -2} should be used
+to indicate FOREVER.
Review comment:
@Apache9 yes this behaviour is not perfect in the sense that both values `0` and `-1` of TTL represents as if TTL was not provided by user.
SnapshotDescription at client side has default value of TTL as -1 and Snapshot proto has ttl's default value as 0. Since we don't want to update default TTL of proto definition (and not make this incompatible change because of this), we thought of keeping special meaning to both values: 0 and -1 (rather than just one of them).
If you really recommend changing behaviour, I would suggest only treating `0` as special (as if TTL was not provided by user so use the config) and avoid `-1`. But so far, as long as users keep above 4 points in mind, snapshot TTL works as expected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3852: HBASE-26458 Update Snapshot TTL doc
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-976028431
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 29s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 4m 2s | master passed |
| +1 :green_heart: | compile | 9m 23s | master passed |
| +1 :green_heart: | checkstyle | 2m 20s | master passed |
| +0 :ok: | refguide | 3m 31s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. |
| +1 :green_heart: | spotbugs | 15m 0s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 53s | the patch passed |
| +1 :green_heart: | compile | 8m 54s | the patch passed |
| +1 :green_heart: | javac | 8m 54s | the patch passed |
| +1 :green_heart: | checkstyle | 1m 56s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +0 :ok: | refguide | 3m 25s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. |
| +1 :green_heart: | hadoopcheck | 19m 14s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. |
| +1 :green_heart: | spotbugs | 14m 43s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 56s | The patch does not generate ASF License warnings. |
| | | 97m 42s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3852 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile refguide |
| uname | Linux ab9684e32957 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 04635f405a |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| refguide | https://nightlies.apache.org/hbase/HBase/HBase-PreCommit-GitHub-PR/PR-3852/3/yetus-general-check/output/branch-site/book.html |
| refguide | https://nightlies.apache.org/hbase/HBase/HBase-PreCommit-GitHub-PR/PR-3852/3/yetus-general-check/output/patch-site/book.html |
| Max. process+thread count | 141 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server . U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/3/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #3852: HBASE-22458 Add UNSET_SNAPSHOT_PROP and fix TTL defaulting
Posted by GitBox <gi...@apache.org>.
virajjasani commented on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-971417508
SnapshotDescription is used at client side to store TTL value. Now, the default value is `-1` if not specified.
Now, while creating snapshot, `SnapshotDescription` is converted to `SnapshotProtos.SnapshotDescription` and here is the logic for TTL conversion:
```
if (snapshotDesc.getTtl() != -1L &&
snapshotDesc.getTtl() < TimeUnit.MILLISECONDS.toSeconds(Long.MAX_VALUE)) {
builder.setTtl(snapshotDesc.getTtl());
}
```
Hence, by default `-1` won't be set in `SnapshotProtos.SnapshotDescription` and the default value of `SnapshotProtos.SnapshotDescription` will be used:
```
message SnapshotDescription {
required string name = 1;
optional string table = 2; // not needed for delete, but checked for in taking snapshot
optional int64 creation_time = 3 [default = 0];
enum Type {
DISABLED = 0;
FLUSH = 1;
SKIPFLUSH = 2;
}
optional Type type = 4 [default = FLUSH];
optional int32 version = 5;
optional string owner = 6;
optional UsersAndPermissions users_and_permissions = 7;
optional int64 ttl = 8 [default = 0];
optional int64 max_file_size = 9 [default = 0];
}
```
As per this proto message, default value of TTL for `SnapshotProtos.SnapshotDescription` would be `0`.
Hence, as per this logic, `0` is `NO_SNAPSHOT_TTL_SPECIFIED` and default TTL will be used that we set with config `hbase.master.snapshot.ttl`.
```
long ttl = snapshot.getTtl();
// set default ttl(sec) if it is not set already or the value is out of the range
if (ttl == SnapshotDescriptionUtils.NO_SNAPSHOT_TTL_SPECIFIED ||
ttl > TimeUnit.MILLISECONDS.toSeconds(Long.MAX_VALUE)) {
final long defaultSnapshotTtl = conf.getLong(HConstants.DEFAULT_SNAPSHOT_TTL_CONFIG_KEY,
HConstants.DEFAULT_SNAPSHOT_TTL);
if (LOG.isDebugEnabled()) {
LOG.debug("Snapshot current TTL value: {} resetting it to default value: {}", ttl,
defaultSnapshotTtl);
}
ttl = defaultSnapshotTtl;
}
SnapshotDescription.Builder builder = snapshot.toBuilder();
builder.setTtl(ttl);
snapshot = builder.build();
```
@joswiatek No specifying TTL while creating snapshot should result in using default TTL as specified by `hbase.master.snapshot.ttl`. For the testing purpose, value of `hbase.master.cleaner.snapshot.interval` can be reduced to test this scenario so that `SnapshotCleanerChore` would be able to delete expired snapshots.
Also, here is the SnapshotCleanerChore logic that would delete only snapshots with value > 0 and value < Long.MAX:
```
long snapshotTtl = snapshotDescription.getTtl();
/*
* Backward compatibility after the patch deployment on HMaster
* Any snapshot with ttl 0 is to be considered as snapshot to keep FOREVER
* Default ttl value specified by {@HConstants.DEFAULT_SNAPSHOT_TTL}
*/
if (snapshotCreatedTime > 0 && snapshotTtl > 0 &&
snapshotTtl < TimeUnit.MILLISECONDS.toSeconds(Long.MAX_VALUE)) {
long currentTime = EnvironmentEdgeManager.currentTime();
if ((snapshotCreatedTime + TimeUnit.SECONDS.toMillis(snapshotTtl)) < currentTime) {
LOG.info("Event: {} Name: {}, CreatedTime: {}, TTL: {}, currentTime: {}",
DELETE_SNAPSHOT_EVENT, snapshotDescription.getName(), snapshotCreatedTime,
snapshotTtl, currentTime);
deleteExpiredSnapshot(snapshotDescription);
}
}
```
I am confident that the current logic should work fine for the case where client doesn't specify snapshot resulting into using TTL specified by config `hbase.master.snapshot.ttl`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani edited a comment on pull request #3852: HBASE-26458 Add UNSET_SNAPSHOT_PROP and fix TTL defaulting
Posted by GitBox <gi...@apache.org>.
virajjasani edited a comment on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-971428634
The only case that might require behaviour change is when user specifies TTL 0:
```
> snapshot 'mytable', 'snapshot1234', {TTL => 0}
```
This snapshot would also be eligible to be deleted as per config `hbase.master.snapshot.ttl` because this config is applicable at entire cluster.
If use specifies TTL value less than -1, then that snapshot will never be deleted based on TTL.
Here is the current behaviour:
1. If client wants to create specific snapshot that should live forever regardless of `hbase.master.snapshot.ttl` config value, then they should specify TTL < -1. e.g `snapshot 'mytable', 'snapshot1234', {TTL => -2}`
2. If we don't want to enable TTL at entire cluster level, no need of any config change because the default value of `hbase.master.snapshot.ttl` is 0, which represents TTL Forever at server side.
It's also important to note that SnapshotCleanerChore can be enabled/disabled using shell command: `snapshot_cleanup_switch`. More details in the section `Enable/Disable Snapshot Auto Cleanup on running cluster` [here](http://hbase.apache.org/book.html#ops.snapshots.takeasnapshot)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #3852: HBASE-26458 Update Snapshot TTL doc
Posted by GitBox <gi...@apache.org>.
virajjasani commented on a change in pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#discussion_r753652443
##########
File path: src/main/asciidoc/_chapters/ops_mgt.adoc
##########
@@ -3150,23 +3150,24 @@ TTL some notion of optional TTL (and optional default TTL) for snapshots could b
hbase> snapshot 'mytable', 'snapshot1234', {TTL => 86400}
----
-The above command creates snapshot `snapshot1234` with TTL of 86400 sec(24 hours)
+The above command creates snapshot `snapshot1234` with TTL of 86400 sec (24 hours)
and hence, the snapshot is supposed to be cleaned up after 24 hours
.Default Snapshot TTL:
-
-- FOREVER by default
-- User specified Default TTL with config `hbase.master.snapshot.ttl`
-
-
-While creating a Snapshot, if TTL in seconds is not specified, by default the snapshot
-would not be deleted automatically. i.e. it would be retained forever until it is
-manually deleted. However, the user can update this default TTL behavior by
-providing default TTL in sec for key: `hbase.master.snapshot.ttl`.
-Value 0 for this config indicates TTL: FOREVER
-
+- User specified default TTL with config `hbase.master.snapshot.ttl`
+- FOREVER if `hbase.master.snapshot.ttl` is not set
+
+While creating a snapshot, if TTL in seconds is not explicitly specified, the above logic will be
+followed to determine the TTL. If no configs are changed, the default behavior is that all snapshots
+will be retained forever (until manual deletion). If a different default TTL behavior is desired,
+`hbase.master.snapshot.ttl` can be set to a default TTL in seconds. Any snapshot created without
+an explicit TTL will take this new value.
+
+NOTE: If `hbase.master.snapshot.ttl` is set, a snapshot with an explicit {TTL => 0} or
+{TTL => -1} will also take this value. In this case, a TTL < -1 (such as {TTL => -2} should be used
+to indicate FOREVER.
Review comment:
@Apache9 yes this behaviour is not perfect in the sense that both values `0` and `-1` of TTL represents as if TTL was not provided by user.
SnapshotDescription at client side has default value of TTL as -1 and Snapshot proto has ttl's default value as 0. Since we don't want to update default TTL of proto definition (and not make this incompatible change because of this), we thought of keeping special meaning to both values: 0 and -1 (rather than just one of them).
If you really recommend changing behaviour, I would suggest only treating `0` as special (as if TTL was not provided by user so use the config) and avoid `-1`. But so far, as long as users keep above 4 points in mind, snapshot TTL works as expected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #3852: HBASE-26458 Update Snapshot TTL doc
Posted by GitBox <gi...@apache.org>.
virajjasani commented on a change in pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#discussion_r753019998
##########
File path: src/main/asciidoc/_chapters/ops_mgt.adoc
##########
@@ -3150,23 +3150,24 @@ TTL some notion of optional TTL (and optional default TTL) for snapshots could b
hbase> snapshot 'mytable', 'snapshot1234', {TTL => 86400}
----
-The above command creates snapshot `snapshot1234` with TTL of 86400 sec(24 hours)
+The above command creates snapshot `snapshot1234` with TTL of 86400 sec (24 hours)
and hence, the snapshot is supposed to be cleaned up after 24 hours
.Default Snapshot TTL:
-
-- FOREVER by default
-- User specified Default TTL with config `hbase.master.snapshot.ttl`
-
-
-While creating a Snapshot, if TTL in seconds is not specified, by default the snapshot
-would not be deleted automatically. i.e. it would be retained forever until it is
-manually deleted. However, the user can update this default TTL behavior by
-providing default TTL in sec for key: `hbase.master.snapshot.ttl`.
-Value 0 for this config indicates TTL: FOREVER
-
+- User specified default TTL with config `hbase.master.snapshot.ttl`
+- FOREVER if `hbase.master.snapshot.ttl` is not set
+
+While creating a snapshot, if TTL in seconds is not explicitly specified, the above logic will be
+followed to determine the TTL. If no configs are changed, the default behavior is that all snapshots
+will be retained forever (until manual deletion). If a different default TTL behavior is desired,
+`hbase.master.snapshot.ttl` can be set to a default TTL in seconds. Any snapshot created without
+an explicit TTL will take this new value.
+
+NOTE: If `hbase.master.snapshot.ttl` is set, a snapshot with an explicit {TTL => 0} or
+{TTL => -1} will also take this value. In this case, a TTL < -1 (such as {TTL => -2} should be used
+to indicate FOREVER.
Review comment:
Thanks @joswiatek.
Looks good. After this line, shall we sum up with these points? Or maybe it's only me thinking that the points would get more attention than description and notes above?
```
To sum-up,
1. Snapshot with TTL value < -1 will stay forever regardless of any server side config changes (until deleted manually by user).
2. Snapshot with TTL value > 0 will be deleted soon after TTL expires.
3. Snapshot created without specifying TTL will always have TTL value represented by config `hbase.master.snapshot.ttl`. Default value of this config is 0, which represents: keep the snapshot forever (until deleted manually by user).
4. From client side, TTL value 0 or -1 should not be provided because they will be treated same as snapshot without TTL (same as above point 3) and hence will use TTL as per value represented by config `hbase.master.snapshot.ttl`.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #3852: HBASE-26458 Update Snapshot TTL doc
Posted by GitBox <gi...@apache.org>.
Apache9 commented on a change in pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#discussion_r753632040
##########
File path: src/main/asciidoc/_chapters/ops_mgt.adoc
##########
@@ -3150,23 +3150,24 @@ TTL some notion of optional TTL (and optional default TTL) for snapshots could b
hbase> snapshot 'mytable', 'snapshot1234', {TTL => 86400}
----
-The above command creates snapshot `snapshot1234` with TTL of 86400 sec(24 hours)
+The above command creates snapshot `snapshot1234` with TTL of 86400 sec (24 hours)
and hence, the snapshot is supposed to be cleaned up after 24 hours
.Default Snapshot TTL:
-
-- FOREVER by default
-- User specified Default TTL with config `hbase.master.snapshot.ttl`
-
-
-While creating a Snapshot, if TTL in seconds is not specified, by default the snapshot
-would not be deleted automatically. i.e. it would be retained forever until it is
-manually deleted. However, the user can update this default TTL behavior by
-providing default TTL in sec for key: `hbase.master.snapshot.ttl`.
-Value 0 for this config indicates TTL: FOREVER
-
+- User specified default TTL with config `hbase.master.snapshot.ttl`
+- FOREVER if `hbase.master.snapshot.ttl` is not set
+
+While creating a snapshot, if TTL in seconds is not explicitly specified, the above logic will be
+followed to determine the TTL. If no configs are changed, the default behavior is that all snapshots
+will be retained forever (until manual deletion). If a different default TTL behavior is desired,
+`hbase.master.snapshot.ttl` can be set to a default TTL in seconds. Any snapshot created without
+an explicit TTL will take this new value.
+
+NOTE: If `hbase.master.snapshot.ttl` is set, a snapshot with an explicit {TTL => 0} or
+{TTL => -1} will also take this value. In this case, a TTL < -1 (such as {TTL => -2} should be used
+to indicate FOREVER.
Review comment:
The behavior is a bit strange, is this by design? Or it just works like this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #3852: HBASE-22458 Add UNSET_SNAPSHOT_PROP and fix TTL defaulting
Posted by GitBox <gi...@apache.org>.
virajjasani commented on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-971405144
Oh wait, now I recall why client and server side have different default TTL values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani edited a comment on pull request #3852: HBASE-26458 Add UNSET_SNAPSHOT_PROP and fix TTL defaulting
Posted by GitBox <gi...@apache.org>.
virajjasani edited a comment on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-971465945
I just tested on the latest HEAD of master branch.
**Configs:**
```
<property>
<name>hbase.master.cleaner.snapshot.interval</name>
<value>10000</value>
</property>
```
**Shell commands and respective outputs:**
> snapshot 't1', 's2'
Took 2.1034 seconds
> snapshot 't1', 's3', {TTL => -2}
Took 1.0925 seconds
=> [{"TTL"=>-2}]
> snapshot 't1', 's1', {TTL => 5}
Took 2.1023 seconds
=> [{"TTL"=>5}]
<img width="1630" alt="Screenshot 2021-11-17 at 4 17 11 PM" src="https://user-images.githubusercontent.com/34790606/142187725-e0f67af3-90bc-47ef-bd28-f73285c27dd3.png">
After 5sec, the snapshot was gone.
<img width="1568" alt="Screenshot 2021-11-17 at 4 19 32 PM" src="https://user-images.githubusercontent.com/34790606/142187833-749fb7a2-5a31-4956-9f65-b7fb8bda36ee.png">
**Configs:**
```
<property>
<name>hbase.master.cleaner.snapshot.interval</name>
<value>10000</value>
</property>
<property>
<name>hbase.master.snapshot.ttl</name>
<value>20</value>
</property>
```
**Shell commands and respective outputs:**
> snapshot 't1', 's4'
Took 2.7291 seconds
Snapshot 's4' gets created with TTL 20 sec because of config `hbase.master.snapshot.ttl`:
<img width="1584" alt="Screenshot 2021-11-17 at 4 21 49 PM" src="https://user-images.githubusercontent.com/34790606/142188010-15a2abf1-26aa-4553-a803-ec2e96b5803d.png">
After 20s, it was gone:
<img width="1539" alt="Screenshot 2021-11-17 at 4 22 04 PM" src="https://user-images.githubusercontent.com/34790606/142188219-1ada2b96-0a64-4f6c-b5f2-711b3abfc8e8.png">
> snapshot 't1', 's5', {TTL => -10}
Took 2.1079 seconds
=> [{"TTL"=>-10}]
<img width="1618" alt="Screenshot 2021-11-17 at 4 23 43 PM" src="https://user-images.githubusercontent.com/34790606/142188278-ed70511c-bea6-44f8-a3d6-c70c674bbd57.png">
1. TTL with negative values < -1, will stay forever regardless of any server side config changes.
2. TTL with positive values will be gone after TTL expires.
3. Snapshot created without specifying TTL will always have TTL value represented by config `hbase.master.snapshot.ttl`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani edited a comment on pull request #3852: HBASE-26458 Add UNSET_SNAPSHOT_PROP and fix TTL defaulting
Posted by GitBox <gi...@apache.org>.
virajjasani edited a comment on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-971465945
I just tested on the latest HEAD of master branch.
**Configs:**
```
<property>
<name>hbase.master.cleaner.snapshot.interval</name>
<value>10000</value>
</property>
```
**Shell commands and respective outputs:**
> snapshot 't1', 's2'
Took 2.1034 seconds
> snapshot 't1', 's3', {TTL => -2}
Took 1.0925 seconds
=> [{"TTL"=>-2}]
> snapshot 't1', 's1', {TTL => 5}
Took 2.1023 seconds
=> [{"TTL"=>5}]
<img width="1630" alt="Screenshot 2021-11-17 at 4 17 11 PM" src="https://user-images.githubusercontent.com/34790606/142187725-e0f67af3-90bc-47ef-bd28-f73285c27dd3.png">
After 5sec, the snapshot was gone.
<img width="1568" alt="Screenshot 2021-11-17 at 4 19 32 PM" src="https://user-images.githubusercontent.com/34790606/142187833-749fb7a2-5a31-4956-9f65-b7fb8bda36ee.png">
**Configs:**
```
<property>
<name>hbase.master.cleaner.snapshot.interval</name>
<value>10000</value>
</property>
<property>
<name>hbase.master.snapshot.ttl</name>
<value>20</value>
</property>
```
**Shell commands and respective outputs:**
> snapshot 't1', 's4'
Took 2.7291 seconds
Snapshot 's4' gets created with TTL 20 sec because of config `hbase.master.snapshot.ttl`:
<img width="1584" alt="Screenshot 2021-11-17 at 4 21 49 PM" src="https://user-images.githubusercontent.com/34790606/142188010-15a2abf1-26aa-4553-a803-ec2e96b5803d.png">
After 20s, it was gone:
<img width="1539" alt="Screenshot 2021-11-17 at 4 22 04 PM" src="https://user-images.githubusercontent.com/34790606/142188219-1ada2b96-0a64-4f6c-b5f2-711b3abfc8e8.png">
> snapshot 't1', 's5', {TTL => -10}
Took 2.1079 seconds
=> [{"TTL"=>-10}]
<img width="1618" alt="Screenshot 2021-11-17 at 4 23 43 PM" src="https://user-images.githubusercontent.com/34790606/142188278-ed70511c-bea6-44f8-a3d6-c70c674bbd57.png">
1. TTL with negative values < `-1`, will stay forever regardless of any server side config changes.
2. TTL with positive values will be gone after TTL expires.
3. Snapshot created without specifying TTL will always have TTL value represented by config `hbase.master.snapshot.ttl`.
4. From client side, TTL values `0` or `-1` should not be provided because they will be treated as snapshot without TTL (same as point `3` above) and hence will represent TTL as per value represented by config `hbase.master.snapshot.ttl`.
I think we have doc with details of server side configs `hbase.master.cleaner.snapshot.interval` and `hbase.master.snapshot.ttl`. From client viewpoint, we should mention above 4 points.
Also, it's bit complex to resolve point `4` completely because in order to represent value as default TTL, we will have to either sacrifice the meaning of 0 or -1 (if not both) in the best case. I think as long as the above mentioned points are taken care of, we should not see any TTL related issues.
Note: updating default value in proto message is considered as incompatible change. Even if we want to present only either 0 or 1 as default TTL, we will need changes to proto definition and be careful of the issues that could arise from rolling upgrade. Given the incompatibility cost, I believe we should rather follow the behaviour mentioned in above 4 points :( Thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani edited a comment on pull request #3852: HBASE-26458 Add UNSET_SNAPSHOT_PROP and fix TTL defaulting
Posted by GitBox <gi...@apache.org>.
virajjasani edited a comment on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-971465945
I just tested on the latest HEAD of master branch.
**Configs:**
<property>
<name>hbase.master.cleaner.snapshot.interval</name>
<value>10000</value>
</property>
**Shell commands and respective outputs:**
> snapshot 't1', 's2'
Took 2.1034 seconds
> snapshot 't1', 's3', {TTL => -2}
Took 1.0925 seconds
=> [{"TTL"=>-2}]
> snapshot 't1', 's1', {TTL => 5}
Took 2.1023 seconds
=> [{"TTL"=>5}]
<img width="1630" alt="Screenshot 2021-11-17 at 4 17 11 PM" src="https://user-images.githubusercontent.com/34790606/142187725-e0f67af3-90bc-47ef-bd28-f73285c27dd3.png">
After 5sec, the snapshot was gone.
<img width="1568" alt="Screenshot 2021-11-17 at 4 19 32 PM" src="https://user-images.githubusercontent.com/34790606/142187833-749fb7a2-5a31-4956-9f65-b7fb8bda36ee.png">
**Configs:**
<property>
<name>hbase.master.cleaner.snapshot.interval</name>
<value>10000</value>
</property>
<property>
<name>hbase.master.snapshot.ttl</name>
<value>20</value>
</property>
**Shell commands and respective outputs:**
> snapshot 't1', 's4'
Took 2.7291 seconds
Snapshot 's4' gets created with TTL 20 sec because of config `hbase.master.snapshot.ttl`:
<img width="1584" alt="Screenshot 2021-11-17 at 4 21 49 PM" src="https://user-images.githubusercontent.com/34790606/142188010-15a2abf1-26aa-4553-a803-ec2e96b5803d.png">
After 20s, it was gone:
<img width="1539" alt="Screenshot 2021-11-17 at 4 22 04 PM" src="https://user-images.githubusercontent.com/34790606/142188219-1ada2b96-0a64-4f6c-b5f2-711b3abfc8e8.png">
> snapshot 't1', 's5', {TTL => -10}
Took 2.1079 seconds
=> [{"TTL"=>-10}]
<img width="1618" alt="Screenshot 2021-11-17 at 4 23 43 PM" src="https://user-images.githubusercontent.com/34790606/142188278-ed70511c-bea6-44f8-a3d6-c70c674bbd57.png">
1. TTL with negative values will stay forever regardless of any server side config changes.
2. TTL with positive values will be gone after TTL expires.
3. Snapshot created without specifying TTL will always have TTL values represented by config `hbase.master.snapshot.ttl`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3852: HBASE-26458 Update Snapshot TTL doc
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-973323831
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 29s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 4m 50s | master passed |
| +1 :green_heart: | compile | 3m 13s | master passed |
| +1 :green_heart: | shadedjars | 8m 22s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 4m 19s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 26s | the patch passed |
| +1 :green_heart: | compile | 3m 12s | the patch passed |
| +1 :green_heart: | javac | 3m 12s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 15s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 4m 25s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 207m 40s | root in the patch passed. |
| | | 253m 39s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3852 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 4770328ba33d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / b2571df7ae |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/2/testReport/ |
| Max. process+thread count | 7568 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server . U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3852: HBASE-26458 Update Snapshot TTL doc
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-976118025
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 32s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 5m 6s | master passed |
| +1 :green_heart: | compile | 3m 31s | master passed |
| +1 :green_heart: | shadedjars | 9m 7s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 5m 0s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 5m 2s | the patch passed |
| +1 :green_heart: | compile | 3m 36s | the patch passed |
| +1 :green_heart: | javac | 3m 36s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 4s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 4m 55s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 213m 21s | root in the patch passed. |
| | | 264m 50s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3852 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 08b0618e844b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 04635f405a |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/3/testReport/ |
| Max. process+thread count | 6845 (vs. ulimit of 30000) |
| modules | C: hbase-common hbase-client hbase-server . U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/3/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #3852: HBASE-22458 Add UNSET_SNAPSHOT_PROP and fix TTL defaulting
Posted by GitBox <gi...@apache.org>.
virajjasani commented on pull request #3852:
URL: https://github.com/apache/hbase/pull/3852#issuecomment-971256634
@joswiatek Since attached patches over the Jira no longer triggers QA, can you once create PRs for branch-2 and branch-1? Once we have the QA, all PRs will be merged and branch-2 change can be cherry-picked to 2.4 and 2.3 release branches.
> I think in the release note, we'b better mention clearly about the broken behavior in the past, and how it is changed after this fix.
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org