You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ignite.apache.org by ab...@apache.org on 2020/08/20 14:56:02 UTC
[ignite] branch IGNITE-7595 updated (6b9c7f4 -> 05b9979)
This is an automated email from the ASF dual-hosted git repository.
abudnikov pushed a change to branch IGNITE-7595
in repository https://gitbox.apache.org/repos/asf/ignite.git.
from 6b9c7f4 update tracing page
new 54102b0 update the tracing page
new 42f88e7 update incorrect version
new b5e4078f add description of tracing configuration command in control script
new 5a702b2 add initial page for cluster states
new 05b9979 minor updates
The 5 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/_data/toc.yaml | 4 +-
docs/_docs/baseline-topology.adoc | 6 +-
docs/_docs/cluster-states.adoc | 21 +++++++
docs/_docs/code-snippets/java/pom.xml | 4 +-
.../java/org/apache/ignite/snippets/Tracing.java | 49 +++++++++++++--
docs/_docs/control-script.adoc | 64 +++++++++++++++++--
docs/_docs/images/span.png | Bin 0 -> 34434 bytes
docs/_docs/images/trace_in_zipkin.png | Bin 120344 -> 118677 bytes
docs/_docs/monitoring-metrics/tracing.adoc | 69 ++++++++++++++++-----
docs/_docs/preface.adoc | 2 +-
docs/_docs/restapi.adoc | 6 +-
docs/_docs/sql-reference/operational-commands.adoc | 2 +-
12 files changed, 191 insertions(+), 36 deletions(-)
create mode 100644 docs/_docs/cluster-states.adoc
create mode 100644 docs/_docs/images/span.png
[ignite] 04/05: add initial page for cluster states
Posted by ab...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
abudnikov pushed a commit to branch IGNITE-7595
in repository https://gitbox.apache.org/repos/asf/ignite.git
commit 5a702b24247a3c7d5d43a501809bde6711365364
Author: abudnikov <ab...@gridgain.com>
AuthorDate: Thu Aug 20 17:55:17 2020 +0300
add initial page for cluster states
---
docs/_docs/cluster-states.adoc | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/docs/_docs/cluster-states.adoc b/docs/_docs/cluster-states.adoc
new file mode 100644
index 0000000..64279ca
--- /dev/null
+++ b/docs/_docs/cluster-states.adoc
@@ -0,0 +1,21 @@
+= Cluster States
+
+
+You cluster can be in one of the three states.
+
+
+* `INACTIVE`: All operations are prohibited.
++
+--
+The default state of the cluster when you start it for the first time
+
+When you first start your cluster, pure in-memory cluster (no persistent data regions)
+
+When you change the cluster state from active to `INACTIVE` (deactivation), the cluster deallocates all memory resources.
+
+include::includes/note-on-deactivation.adoc[]
+
+--
+* `ACTIVE`: This is the normal mode of the cluster. You can change change the cluster topology and execute any operations.
+
+* `ACTIVE_READ_ONLY`: The read-only mode. The operations that modify data in the cluster are prohibited.
[ignite] 05/05: minor updates
Posted by ab...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
abudnikov pushed a commit to branch IGNITE-7595
in repository https://gitbox.apache.org/repos/asf/ignite.git
commit 05b99791d42e9e5941a4b08f1708a96f8418ea4f
Author: abudnikov <ab...@gridgain.com>
AuthorDate: Thu Aug 20 17:55:38 2020 +0300
minor updates
---
docs/_data/toc.yaml | 4 +++-
docs/_docs/code-snippets/java/pom.xml | 4 ++--
docs/_docs/preface.adoc | 2 +-
docs/_docs/restapi.adoc | 6 +++---
docs/_docs/sql-reference/operational-commands.adoc | 2 +-
5 files changed, 10 insertions(+), 8 deletions(-)
diff --git a/docs/_data/toc.yaml b/docs/_data/toc.yaml
index 91c80c6..1a412ff 100644
--- a/docs/_data/toc.yaml
+++ b/docs/_data/toc.yaml
@@ -109,6 +109,8 @@
url: /persistence/persistence-tuning
- title: Baseline Topology
url: /baseline-topology
+- title: Cluster States
+ url: /cluster-states
- title: Data Rebalancing
url: /data-rebalancing
- title: Partition Loss Policy
@@ -179,7 +181,7 @@
url: /near-cache
# - title: .NET Platform Cache
# url: /platform-cache
-- title: Metrics and Monitoring
+- title: Monitoring
items:
- title: Introduction
url: /monitoring-metrics/intro
diff --git a/docs/_docs/code-snippets/java/pom.xml b/docs/_docs/code-snippets/java/pom.xml
index 94065a7..5eef9f1 100644
--- a/docs/_docs/code-snippets/java/pom.xml
+++ b/docs/_docs/code-snippets/java/pom.xml
@@ -57,7 +57,7 @@
<version>${ignite.version}</version>
</dependency>
- <dependency>
+ <dependency>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-opencensus</artifactId>
<version>${ignite.version}</version>
@@ -73,7 +73,7 @@
<artifactId>ignite-aws</artifactId>
<version>${ignite.version}</version>
</dependency>
- <dependency>
+ <dependency>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-compress</artifactId>
<version>${ignite.version}</version>
diff --git a/docs/_docs/preface.adoc b/docs/_docs/preface.adoc
index 22b443b..1d03cca 100644
--- a/docs/_docs/preface.adoc
+++ b/docs/_docs/preface.adoc
@@ -9,5 +9,5 @@ include::includes/intro-languages.adoc[]
== Ignite Examples
The Apache Ignite github repository contains a number of runnable examples that illustrate various Ignite functionality.
-Refer to the link:{githubUrl}/examples[Examples] folder in the Ignite code base.
+Refer to the link:{githubUrl}/examples[Examples^] folder in the Ignite code base.
The examples are also available in the binary distribution.
diff --git a/docs/_docs/restapi.adoc b/docs/_docs/restapi.adoc
index 1dab3f2..bcc0cc2 100644
--- a/docs/_docs/restapi.adoc
+++ b/docs/_docs/restapi.adoc
@@ -2,7 +2,7 @@
:request_table_props: cols="15%,10%,10%,45%,20%",options="header"
:response_table_props: cols="15%,15%,50%,20%",options="header"
-Ignite provides an HTTP REST client that gives you the ability to communicate with the grid over HTTP and HTTPS protocols using the REST approach. REST APIs can be used to perform different operations like read/write from/to cache, execute tasks, get various metrics, and more.
+Ignite provides an HTTP REST client that can communicate with the cluster over HTTP and HTTPS protocols using the REST approach. REST APIs can be used to perform different operations like read/write from/to cache, execute tasks, get various metrics, and more.
Internally, Ignite uses Jetty to provide HTTP server features. See <<Configuration>> section below for details on how to configure jetty.
@@ -72,7 +72,7 @@ The following table describes the properties of `ConnectorConfiguration` that ar
|`null`
|`setMessageInterceptor(...)`
-|The interceptor provides the ability to transform all objects exchanged via REST protocol. For example, if you use custom serialisation on client you can write an interceptor to transform binary representations received from the client to Java objects and later access them from Java code directly.
+|The interceptor transforms all objects exchanged via REST protocol. For example, if you use custom serialisation on client you can write an interceptor to transform binary representations received from the client to Java objects and later access them from Java code directly.
|Yes
|`null`
|=======
@@ -169,7 +169,7 @@ To set a custom expire time, set the system variable: `IGNITE_REST_SESSION_TIMEO
====
== Data Types
-The REST API also provides support for Java built-in types for put/get operations via `keyType` and `valueType` optional parameters.
+The REST API also supports Java built-in types for put/get operations via `keyType` and `valueType` optional parameters.
Note that unless one of the below mentioned types are explicitly specified, the REST protocol exchanges the key-value data in `String` format.
This means that the data is stored and retrieved to/from the cluster as a `String`.
diff --git a/docs/_docs/sql-reference/operational-commands.adoc b/docs/_docs/sql-reference/operational-commands.adoc
index 262a2b7..482f784 100644
--- a/docs/_docs/sql-reference/operational-commands.adoc
+++ b/docs/_docs/sql-reference/operational-commands.adoc
@@ -1,7 +1,7 @@
= Operational Commands
-The following operational commands are supported by Ignite:
+Ignite supports the following operational commands:
== COPY
[ignite] 01/05: update the tracing page
Posted by ab...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
abudnikov pushed a commit to branch IGNITE-7595
in repository https://gitbox.apache.org/repos/asf/ignite.git
commit 54102b07d040cc3a10d5b0ba819c8945106c7748
Author: abudnikov <ab...@gridgain.com>
AuthorDate: Thu Aug 20 17:48:59 2020 +0300
update the tracing page
---
.../java/org/apache/ignite/snippets/Tracing.java | 49 +++++++++++++--
docs/_docs/images/span.png | Bin 0 -> 34434 bytes
docs/_docs/images/trace_in_zipkin.png | Bin 120344 -> 118677 bytes
docs/_docs/monitoring-metrics/tracing.adoc | 69 ++++++++++++++++-----
4 files changed, 98 insertions(+), 20 deletions(-)
diff --git a/docs/_docs/code-snippets/java/src/main/java/org/apache/ignite/snippets/Tracing.java b/docs/_docs/code-snippets/java/src/main/java/org/apache/ignite/snippets/Tracing.java
index 61ec931..8f676f5 100644
--- a/docs/_docs/code-snippets/java/src/main/java/org/apache/ignite/snippets/Tracing.java
+++ b/docs/_docs/code-snippets/java/src/main/java/org/apache/ignite/snippets/Tracing.java
@@ -1,13 +1,19 @@
package org.apache.ignite.snippets;
import org.apache.ignite.Ignite;
+import org.apache.ignite.IgniteCache;
+import org.apache.ignite.IgniteTransactions;
import org.apache.ignite.Ignition;
+import org.apache.ignite.cache.CacheAtomicityMode;
+import org.apache.ignite.configuration.CacheConfiguration;
import org.apache.ignite.configuration.IgniteConfiguration;
import org.apache.ignite.spi.tracing.Scope;
import org.apache.ignite.spi.tracing.TracingConfigurationCoordinates;
import org.apache.ignite.spi.tracing.TracingConfigurationParameters;
+import org.apache.ignite.transactions.Transaction;
import org.junit.jupiter.api.Test;
+import io.opencensus.common.Duration;
import io.opencensus.exporter.trace.zipkin.ZipkinExporterConfiguration;
import io.opencensus.exporter.trace.zipkin.ZipkinTraceExporter;
@@ -26,6 +32,7 @@ public class Tracing {
ignite.close();
}
+ @Test
void enableSampling() {
//tag::enable-sampling[]
Ignite ignite = Ignition.start();
@@ -35,21 +42,55 @@ public class Tracing {
new TracingConfigurationParameters.Builder().withSamplingRate(0.5).build());
//end::enable-sampling[]
+ ignite.close();
}
@Test
void exportToZipkin() {
//tag::export-to-zipkin[]
+
+ //register Zipkin exporter
ZipkinTraceExporter.createAndRegister(
ZipkinExporterConfiguration.builder().setV2Url("http://localhost:9411/api/v2/spans")
.setServiceName("ignite-cluster").build());
- IgniteConfiguration cfg = new IgniteConfiguration();
-
- cfg.setTracingSpi(new org.apache.ignite.spi.tracing.opencensus.OpenCensusTracingSpi());
+ IgniteConfiguration cfg = new IgniteConfiguration().setClientMode(true)
+ .setTracingSpi(new org.apache.ignite.spi.tracing.opencensus.OpenCensusTracingSpi());
Ignite ignite = Ignition.start(cfg);
- //end::config[]
+
+ //enable trace sampling for transactions with 100% sampling rate
+ ignite.tracingConfiguration().set(
+ new TracingConfigurationCoordinates.Builder(Scope.TX).build(),
+ new TracingConfigurationParameters.Builder().withSamplingRate(1).build());
+
+ //create a transactional cache
+ IgniteCache<Integer, String> cache = ignite
+ .getOrCreateCache(new CacheConfiguration<Integer, String>("myCache")
+ .setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL));
+
+ IgniteTransactions transactions = ignite.transactions();
+
+ // start a transaction
+ try (Transaction tx = transactions.txStart()) {
+ //do some operations
+ cache.put(1, "test value");
+
+ System.out.println(cache.get(1));
+
+ cache.put(1, "second value");
+
+ tx.commit();
+ }
+
+ try {
+ //This code here is to wait until the trace is exported to Zipkin.
+ //If your application doesn't stop here, you don't need this piece of code.
+ Thread.sleep(5_000);
+ } catch (InterruptedException e) {
+ e.printStackTrace();
+ }
+
//end::export-to-zipkin[]
ignite.close();
}
diff --git a/docs/_docs/images/span.png b/docs/_docs/images/span.png
new file mode 100644
index 0000000..ae05b72
Binary files /dev/null and b/docs/_docs/images/span.png differ
diff --git a/docs/_docs/images/trace_in_zipkin.png b/docs/_docs/images/trace_in_zipkin.png
index fdf1e18..074b264 100644
Binary files a/docs/_docs/images/trace_in_zipkin.png and b/docs/_docs/images/trace_in_zipkin.png differ
diff --git a/docs/_docs/monitoring-metrics/tracing.adoc b/docs/_docs/monitoring-metrics/tracing.adoc
index 5536387..1208fb0 100644
--- a/docs/_docs/monitoring-metrics/tracing.adoc
+++ b/docs/_docs/monitoring-metrics/tracing.adoc
@@ -2,12 +2,12 @@
:javaFile: {javaCodeDir}/Tracing.java
-WARNING: Experimental
+WARNING: This feature is experimental.
-A number of APIs in Ignite are instrumented with OpenCensus for tracing.
-You can collect distributed traces of various tasks execuded in your cluster and use this information to diagnose latency problems.
+A number of APIs in Ignite are instrumented for tracing with OpenCensus.
+You can collect distributed traces of various tasks executed in your cluster and use this information to diagnose latency problems.
-We suggest you get familiar with OpenCensus tracing documentation: https://opencensus.io/tracing/[^].
+We suggest you get familiar with OpenCensus tracing documentation before reading this chapter: https://opencensus.io/tracing/[^].
The following Ignite APIs are instrumented for tracing:
@@ -16,6 +16,12 @@ The following Ignite APIs are instrumented for tracing:
* Exchange
* Transactions
+
+To view traces, you must export them into an external system.
+You can use one of the OpenCensus exporters or write your own, but in any case, you will have to write code that registers an exporter in Ignite.
+Refer to <<Exporting Traces>> for details.
+
+
== Configuring Tracing
Enable OpenCensus tracing in the node configuration. All nodes in the cluster must use the same tracing configuration.
@@ -39,11 +45,10 @@ tab:C++[unsupported]
--
-
== Enabling Sampling of Traces
-When you start your cluster with the above configuration, Ignite will not collect traces.
-You have to enable collection of traces for a specific API at runtime.
+When you start your cluster with the above configuration, Ignite does not collect traces.
+You have to enable trace sampling for a specific API at runtime.
You can turn trace sampling on and off at will, for example, only for the period when you are troubleshooting a problem.
You can do this in two ways:
@@ -54,7 +59,7 @@ You can do this in two ways:
Traces are collected with a given probabilistic sampling rate.
The rate is specified as a value between 0 and 1 inclusive: `0` means no sampling, `1` means always sampling.
-When the sampling rate is set to a value greater than 0, Ignite will collect traces.
+When the sampling rate is set to a value greater than 0, Ignite collects traces.
To disable trace collection, set the sampling rate to 0.
The following sections describe the two ways of enabling trace sampling.
@@ -100,30 +105,62 @@ The `--sampling-rate` is the probabilistic sampling rate, a number between `0` a
* `1` means always sampling.
-== Viewing Traces
+== Exporting Traces
To view traces, you need to export them to an external backend using one of the available exporters.
-OpenCensus supports a number of exporter out-of-the-box, and you can write a custom exporter.
+OpenCensus supports a number of exporters out-of-the-box, and you can write a custom one.
Refer to the link:https://opencensus.io/exporters/[OpenCensus Exporters^] for details.
In this section, we will show how to export traces to link:https://zipkin.io[Zipkin^].
-Follow link:https://zipkin.io/pages/quickstart.html[this guide^] to launch Zipkin on your machine.
+. Follow link:https://zipkin.io/pages/quickstart.html[this guide^] to launch Zipkin on your machine.
+. Register `ZipkinTraceExporter` in the application where you start Ignite:
++
+--
[source, java]
----
include::{javaFile}[tags=export-to-zipkin, indent=0]
----
+--
-Start one server node and then start another node.
-TODO
+. Open http://localhost:9411/zipkin[^] in your browser and click the search icon.
++
+--
+This is what a trace of the transaction looks like:
-open http://localhost:9411[^] in your browser
+image::images/trace_in_zipkin.png[]
+--
-This is what a trace of a node join event looks like:
+== Analyzing Trace Data
-image::images/trace_in_zipkin.png[]
+A trace is recorded information about the execution of a specific event.
+Each trace consists of a tree of _spans_.
+A span is an individual unit of work performed by the system in order to process the event.
+
+Because of the distributed nature of Ignite, an operation usually involves multiple nodes.
+Therefore, a trace can include spans from multiple nodes.
+Each span always contains the information about the node where the corresponding operation was executed.
+
+In the image of the transaction trace presented above, you can see that the trace contains the spans associated with the following operations:
+
+* acquire locks (`transactions.colocated.lock.map`),
+* get (`transactions.near.enlist.read`),
+* put (`transactions.near.enlist.write`),
+* commit (`transactions.commit`), and
+* close (`transactions.close`).
+
+The commit operation, in turn, consists of two operations: prepare and finish.
+
+You can click on each span to view the annotations and tags attached to it.
+
+//=== Analyzing Spans
+
+image::images/span.png[Span]
+=== Annotations
+=== Tags
+The `node.id` and `node.consistentId` are the ID and consistent ID of the node where the root operation started.
[ignite] 03/05: add description of tracing configuration command in
control script
Posted by ab...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
abudnikov pushed a commit to branch IGNITE-7595
in repository https://gitbox.apache.org/repos/asf/ignite.git
commit b5e4078fde0cbf0ae3324a786e2c4ab0f2601051
Author: abudnikov <ab...@gridgain.com>
AuthorDate: Thu Aug 20 17:54:57 2020 +0300
add description of tracing configuration command in control script
---
docs/_docs/control-script.adoc | 64 ++++++++++++++++++++++++++++++++++++++----
1 file changed, 59 insertions(+), 5 deletions(-)
diff --git a/docs/_docs/control-script.adoc b/docs/_docs/control-script.adoc
index e713ae3..251b6b2 100644
--- a/docs/_docs/control-script.adoc
+++ b/docs/_docs/control-script.adoc
@@ -18,7 +18,6 @@ tab:Windows[]
----
control.bat <connection parameters> <command> <arguments>
----
-
--
== Connecting to Cluster
@@ -168,7 +167,7 @@ Execution time: 333 ms
=== Adding Nodes to Baseline Topology
To add a node to the baseline topology, run the command given below.
-After the node is added, the link:data-rebalancing[rebalancing process] will start.
+After the node is added, the link:data-rebalancing[rebalancing process] starts.
[tabs]
--
@@ -188,8 +187,7 @@ control.bat --baseline add _consistentId1,consistentId2,..._ [--yes]
To remove a node from the baseline topology, use the `remove` command.
Only offline nodes can be removed from the baseline topology: shut down the node first and then use the `remove` command.
-This operation starts the rebalancing process, which will re-distribute the data across the nodes that remain in the baseline topology.
-//The node will not be shut down until you manually do it.
+This operation starts the rebalancing process, which re-distributes the data across the nodes that remain in the baseline topology.
[tabs]
--
@@ -286,6 +284,7 @@ control.bat --baseline auto_adjust disable
----
--
+
== Transaction Management
The control script allows you to get the information about the transactions being executed in the cluster.
@@ -507,5 +506,60 @@ validate_indexes has finished with errors (listed above).
====
[discrete]
=== Cluster Should Be Idle During `validate_indexes` Check
-Like `idle_verify`, indexes validation tool works correctly only if updates are stopped. Otherwise, there may be a race between the checker thread and the thread that updates the entry/index, which can result in a false positive error report.
+Like `idle_verify`, index validation tool works correctly only if updates are stopped. Otherwise, there may be a race between the checker thread and the thread that updates the entry/index, which can result in a false positive error report.
====
+
+
+== Tracing Configuration
+
+You can enable or disable sampling of traces for a specific API by using the `--tracing-configuration` command.
+Refer to the link:monitoring-metrics/tracing[Tracing] section for details.
+
+Before using the command, enable experimental features of the control script:
+
+[source, shell]
+----
+export IGNITE_ENABLE_EXPERIMENTAL_COMMAND=true
+----
+
+To view the current tracing configuration, execute the following command:
+
+[source, shell]
+----
+control.sh --tracing-configuration
+----
+
+To enable trace sampling for a specific API:
+
+
+[source, shell]
+----
+control.sh --tracing-configuration set --scope DISCOVERY|EXCHANGE|COMMUNICATION|TX --sampling-rate <rate> --label <label>
+----
+
+Parameters:
+
+[cols="1,3",opts="header"]
+|===
+| Parameter | Description
+| `--scope` a| The API you want to trace:
+
+* `DISCOVERY`: discovery events
+* `EXCHANGE`: exchange events
+* `COMMUNICATION`: communication events
+* `TX`: transactions
+
+| `--sampling-rate` a| The probabilistic sampling rate, a number between `0.0` and `1.0`, inclusive.
+ `0` means no sampling (default), `1` means always sampling. Ex. `0.5` means every trace is sampled with the probability of 50%.
+
+| `--label` | Only applicable to the `TX` scope. The parameter defines the sampling rate for the transactions with the given label.
+
+|===
+
+
+
+
+
+
+
+
[ignite] 02/05: update incorrect version
Posted by ab...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
abudnikov pushed a commit to branch IGNITE-7595
in repository https://gitbox.apache.org/repos/asf/ignite.git
commit 42f88e7a357d04c5f41e9ff7c05331797d109b29
Author: abudnikov <ab...@gridgain.com>
AuthorDate: Thu Aug 20 17:53:44 2020 +0300
update incorrect version
---
docs/_docs/baseline-topology.adoc | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/docs/_docs/baseline-topology.adoc b/docs/_docs/baseline-topology.adoc
index 91fcb1e..a0d3d98 100644
--- a/docs/_docs/baseline-topology.adoc
+++ b/docs/_docs/baseline-topology.adoc
@@ -34,9 +34,9 @@ In pure in-memory clusters, the default behavior is to adjust the baseline topol
automatically when you add or remove server nodes from the cluster. The data is rebalanced automatically, too.
You can disable the baseline autoadjustment feature and manage baseline topology manually.
-NOTE: In previous releases, baseline topology was relevant only to clusters with persistence. However, since version 8.7.14,
-it applies to in-memory clusters as well. If you have a pure in-memory cluster, the transition should be transparent
-for you because, by default, the baseline topology changes automatically when a server node leaves or joins the cluster.
+NOTE: In previous releases, baseline topology was relevant only to clusters with persistence.
+However, since version 2.8.0, it applies to in-memory clusters as well.
+If you have a pure in-memory cluster, the transition should be transparent for you because, by default, the baseline topology changes automatically when a server node leaves or joins the cluster.
== Baseline Topology in Persistent Clusters