You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ignite.apache.org by mm...@apache.org on 2021/03/16 21:07:56 UTC
[ignite] branch master updated: IGNITE-14247 Document the snapshot
check command (#8878)
This is an automated email from the ASF dual-hosted git repository.
mmuzaf pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/ignite.git
The following commit(s) were added to refs/heads/master by this push:
new eeeb81d IGNITE-14247 Document the snapshot check command (#8878)
eeeb81d is described below
commit eeeb81dc35d1353eb437510a11ddbea2dbcda45d
Author: Maxim Muzafarov <mm...@apache.org>
AuthorDate: Wed Mar 17 00:07:40 2021 +0300
IGNITE-14247 Document the snapshot check command (#8878)
---
docs/Gemfile | 3 ++
docs/_data/toc.yaml | 6 ++--
docs/_docs/persistence/snapshot-directory.adoc | 39 ++++++++++++++++++++++
.../{persistence => snapshots}/snapshots.adoc | 34 +++++++++----------
docs/_docs/tools/control-script.adoc | 19 +++++++++++
5 files changed, 82 insertions(+), 19 deletions(-)
diff --git a/docs/Gemfile b/docs/Gemfile
index f471d02..381249a 100644
--- a/docs/Gemfile
+++ b/docs/Gemfile
@@ -12,3 +12,6 @@ end
gem 'thread_safe', '~> 0.3.6'
gem 'slim', '~> 4.0.1'
gem 'tilt', '~> 2.0.9'
+
+# Ruby 3.0.0 requires dependency which doesn't contains in the bundle
+gem "webrick", "~> 1.7"
diff --git a/docs/_data/toc.yaml b/docs/_data/toc.yaml
index 53eb7f1..0b7a248 100644
--- a/docs/_data/toc.yaml
+++ b/docs/_data/toc.yaml
@@ -139,12 +139,14 @@
url: persistence/swap
- title: Implementing Custom Cache Store
url: persistence/custom-cache-store
- - title: Cluster Snapshots
- url: persistence/snapshots
+ - title: Configuring Snapshot Directory
+ url: persistence/snapshot-directory
- title: Disk Compression
url: persistence/disk-compression
- title: Tuning Persistence
url: persistence/persistence-tuning
+- title: Cluster Snapshots
+ url: snapshots/snapshots
- title: Configuring Caches
items:
- title: Cache Configuration
diff --git a/docs/_docs/persistence/snapshot-directory.adoc b/docs/_docs/persistence/snapshot-directory.adoc
new file mode 100644
index 0000000..a046fab
--- /dev/null
+++ b/docs/_docs/persistence/snapshot-directory.adoc
@@ -0,0 +1,39 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+= Snapshot Directory
+
+== Configuring Snapshot Directory
+
+By default, a segment of the snapshot is stored in the work directory of a respective Ignite node. This segment uses the
+same storage media where Ignite Persistence keeps data, index, WAL, and other files. Since the snapshot can consume as much space as
+already taken by the persistence files and can affect your applications' performance by sharing the disk I/O with the
+Ignite Persistence routines, it's suggested to store the snapshot and persistence files on different media.
+
+You can avoid this interference between Ignite Native persistence and snapshotting
+by either changing link:persistence/native-persistence#configuring-persistent-storage-directory[storage directories of the persistence files]
+or overriding the default snapshots' location as shown below:
+[tabs]
+--
+tab:XML[]
+[source, xml]
+----
+include::code-snippets/xml/snapshots.xml[tags=ignite-config;!discovery, indent=0]
+----
+tab:Java[]
+[source, java]
+----
+include::{javaCodeDir}/Snapshots.java[tags=config, indent=0]
+----
+--
diff --git a/docs/_docs/persistence/snapshots.adoc b/docs/_docs/snapshots/snapshots.adoc
similarity index 89%
rename from docs/_docs/persistence/snapshots.adoc
rename to docs/_docs/snapshots/snapshots.adoc
index b2d345b..a0f244b 100644
--- a/docs/_docs/persistence/snapshots.adoc
+++ b/docs/_docs/snapshots/snapshots.adoc
@@ -83,22 +83,8 @@ media where Ignite Persistence keeps data, index, WAL, and other files. Since th
already taken by the persistence files and can affect your applications' performance by sharing the disk I/O with the
Ignite Persistence routines, it's suggested to store the snapshot and persistence files on different media.
-You can avoid this interference between Ignite Native persistence and snapshotting
-by either changing link:persistence/native-persistence#configuring-persistent-storage-directory[storage directories of the persistence files]
-or overriding the default snapshots' location as shown below:
-[tabs]
---
-tab:XML[]
-[source, xml]
-----
-include::code-snippets/xml/snapshots.xml[tags=ignite-config;!discovery, indent=0]
-----
-tab:Java[]
-[source, java]
-----
-include::{javaCodeDir}/Snapshots.java[tags=config, indent=0]
-----
---
+See the link:persistence/snapshot-directory#configuring-snapshot-directory[Configuring Snapshot Directory] page for
+configuration examples.
== Creating Snapshot
@@ -106,7 +92,7 @@ Ignite provides several APIs for the snapshot creation. Let's review all the opt
=== Using Control Script
-Ignite ships the link:control-script[control script] that supports snapshots-related commands listed below:
+Ignite ships the link:tools/control-script[Control Script] that supports snapshots-related commands listed below:
[source,shell]
----
@@ -145,6 +131,20 @@ include::{javaCodeDir}/Snapshots.java[tags=create, indent=0]
----
--
+== Checking Snapshot Consistency
+
+Usually all the cluster nodes run on different machines and have the snapshot data spread across the cluster.
+Each node stores its own snapshot segment, so in some cases it may be necessary to check the snapshot for completeness
+of data and for data consistency across the cluster before restoring from the snapshot.
+
+For such cases, Apache Ignite is delivered with built-in snapshot consistency check commands that enable you to verify
+internal data consistency, calculate data partitions hashes and pages checksums, and print out the result if a
+problem is found. The check command also compares hashes of a primary partitions with corresponding backup partitions
+and reports any differences.
+
+See the link:tools/control-script#checking-snapshot-consistency[Control Script] that supports snapshots-related checking
+commands.
+
== Restoring From Snapshot
Currently, the data restore procedure has to be performed manually. In a nutshell, you need to stop the cluster,
diff --git a/docs/_docs/tools/control-script.adoc b/docs/_docs/tools/control-script.adoc
index d5ce4fe..b8417ee 100644
--- a/docs/_docs/tools/control-script.adoc
+++ b/docs/_docs/tools/control-script.adoc
@@ -521,6 +521,25 @@ validate_indexes has finished with errors (listed above).
Like `idle_verify`, index validation tool works correctly only if updates are stopped. Otherwise, there may be a race between the checker thread and the thread that updates the entry/index, which can result in a false positive error report.
====
+=== Checking Snapshot Consistency
+
+The checking snapshot consistency command works the same way as the `idle_verify` command does. It compares hashes between
+a primary partition and a corresponding backup partitions and prints a report if any differences are found.
+Differences may be the result of inconsistencies in some data on the cluster from which the snapshot was taken. It is
+recommended to perform the `idle_verify` procedure on the cluster if this case occurs.
+
+This procedure does not require the cluster to be in the `idle` state.
+
+[tabs]
+--
+tab:Shell[]
+[source,shell]
+----
+# Checks that partitions of all snapshot caches have the correct checksums and primary/backup ones actually contain the same data.
+control.(sh|bat) --snapshot check snapshot_name
+----
+--
+
=== Check SQL Index Inline Size
A running Ignite cluster could have different SQL index inline sizes on its cluster nodes.