You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@uniffle.apache.org by zh...@apache.org on 2022/10/25 13:05:42 UTC
[incubator-uniffle-website] branch master updated (d9aba51 -> ac53c8a)
This is an automated email from the ASF dual-hosted git repository.
zhifgli pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-uniffle-website.git
discard d9aba51 Add metrics docs
new ac53c8a Add metrics docs
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (d9aba51)
\
N -- N -- N refs/heads/master (ac53c8a)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/01-intro.md | 17 -----------------
docs/02-client-guide.md | 17 -----------------
docs/03-deploy-uniffle-cluster/00-coordinator-guide.md | 17 -----------------
docs/03-deploy-uniffle-cluster/01-server-guide.md | 17 -----------------
.../02-hardware-provisioning.md | 17 -----------------
.../00-uniffle-operator-design.md | 17 -----------------
docs/04-how-to-deploy-on-k8s/01-install.md | 18 ------------------
docs/04-how-to-deploy-on-k8s/02-examples.md | 18 ------------------
8 files changed, 138 deletions(-)
[incubator-uniffle-website] 01/01: Add metrics docs
Posted by zh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zhifgli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-uniffle-website.git
commit ac53c8a73017a6758d479b40f981b5b84b30657c
Author: frankzfli <fr...@tencent.com>
AuthorDate: Tue Oct 25 20:55:25 2022 +0800
Add metrics docs
---
docs/01-intro.md | 17 -----------------
docs/02-client-guide.md | 17 -----------------
.../00-coordinator-guide.md | 17 -----------------
docs/03-deploy-uniffle-cluster/01-server-guide.md | 17 -----------------
.../02-hardware-provisioning.md | 17 -----------------
.../00-uniffle-operator-design.md | 17 -----------------
docs/04-how-to-deploy-on-k8s/01-install.md | 18 ------------------
docs/04-how-to-deploy-on-k8s/02-examples.md | 18 ------------------
docs/05-metrics.md | 22 ++++++++++++++++++++--
9 files changed, 20 insertions(+), 140 deletions(-)
diff --git a/docs/01-intro.md b/docs/01-intro.md
index 2ae9ca2..3f2b737 100644
--- a/docs/01-intro.md
+++ b/docs/01-intro.md
@@ -1,20 +1,3 @@
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
---
title: intro
sidebar_position: 1
diff --git a/docs/02-client-guide.md b/docs/02-client-guide.md
index dce7120..4bb2946 100644
--- a/docs/02-client-guide.md
+++ b/docs/02-client-guide.md
@@ -1,20 +1,3 @@
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
# Uniffle Shuffle Client Guide
Uniffle is designed as a unified shuffle engine for multiple computing frameworks, including Apache Spark and Apache Hadoop.
diff --git a/docs/03-deploy-uniffle-cluster/00-coordinator-guide.md b/docs/03-deploy-uniffle-cluster/00-coordinator-guide.md
index 0afb99c..0274885 100644
--- a/docs/03-deploy-uniffle-cluster/00-coordinator-guide.md
+++ b/docs/03-deploy-uniffle-cluster/00-coordinator-guide.md
@@ -1,20 +1,3 @@
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
# Uniffle Coordinator Guide
Uniffle is a unified remote shuffle service for compute engines, the role of coordinator is responsibility for
diff --git a/docs/03-deploy-uniffle-cluster/01-server-guide.md b/docs/03-deploy-uniffle-cluster/01-server-guide.md
index a11c339..6a00a00 100644
--- a/docs/03-deploy-uniffle-cluster/01-server-guide.md
+++ b/docs/03-deploy-uniffle-cluster/01-server-guide.md
@@ -1,20 +1,3 @@
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
# Uniffle Shuffle Server Guide
## Deploy
diff --git a/docs/03-deploy-uniffle-cluster/02-hardware-provisioning.md b/docs/03-deploy-uniffle-cluster/02-hardware-provisioning.md
index 66b2951..d7182a2 100644
--- a/docs/03-deploy-uniffle-cluster/02-hardware-provisioning.md
+++ b/docs/03-deploy-uniffle-cluster/02-hardware-provisioning.md
@@ -1,20 +1,3 @@
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
# HARDWARE PROVISIONING
A common question received by Uniffle users is how to configure hardware for it. While the right hardware will depend on the situation, we make the following recommendations.
diff --git a/docs/04-how-to-deploy-on-k8s/00-uniffle-operator-design.md b/docs/04-how-to-deploy-on-k8s/00-uniffle-operator-design.md
index 13c81da..fb57447 100644
--- a/docs/04-how-to-deploy-on-k8s/00-uniffle-operator-design.md
+++ b/docs/04-how-to-deploy-on-k8s/00-uniffle-operator-design.md
@@ -1,20 +1,3 @@
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
# Uniffle Operator Design
## Summary
diff --git a/docs/04-how-to-deploy-on-k8s/01-install.md b/docs/04-how-to-deploy-on-k8s/01-install.md
index 82b8930..b6c4fb4 100644
--- a/docs/04-how-to-deploy-on-k8s/01-install.md
+++ b/docs/04-how-to-deploy-on-k8s/01-install.md
@@ -1,21 +1,3 @@
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-
# Installation
This section shows us how to install operator in our cluster.
diff --git a/docs/04-how-to-deploy-on-k8s/02-examples.md b/docs/04-how-to-deploy-on-k8s/02-examples.md
index f8514c0..d18939c 100644
--- a/docs/04-how-to-deploy-on-k8s/02-examples.md
+++ b/docs/04-how-to-deploy-on-k8s/02-examples.md
@@ -1,21 +1,3 @@
-<!--
- ~ Licensed to the Apache Software Foundation (ASF) under one or more
- ~ contributor license agreements. See the NOTICE file distributed with
- ~ this work for additional information regarding copyright ownership.
- ~ The ASF licenses this file to You under the Apache License, Version 2.0
- ~ (the "License"); you may not use this file except in compliance with
- ~ the License. You may obtain a copy of the License at
- ~
- ~ http://www.apache.org/licenses/LICENSE-2.0
- ~
- ~ Unless required by applicable law or agreed to in writing, software
- ~ distributed under the License is distributed on an "AS IS" BASIS,
- ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- ~ See the License for the specific language governing permissions and
- ~ limitations under the License.
- -->
-
-
# Examples
We need to create configMap first which saves coordinators, shuffleServers and log4j's configuration(we can refer
diff --git a/docs/05-metrics.md b/docs/05-metrics.md
index aac7ffe..f152f76 100644
--- a/docs/05-metrics.md
+++ b/docs/05-metrics.md
@@ -1,3 +1,21 @@
-# Import Metrics
+# Metrics System
-for test
\ No newline at end of file
+In Uniffle, both coordinator and shuffle server have provided many runtime metrics, which help to maintain the Uniffle cluster in production.
+
+We list a [script](https://github.com/apache/incubator-uniffle/blob/master/bin/get-metrics.py) to get these metrics via http protocol.
+
+```
+python get-metrics.py [serverListFile] [jettyPort] [metricName]
+```
+
+Extract server ip from line in the serverListFile, which contains lines of server node and the format is $IP-$PORT.
+The ip and port are server's rpc ip and port respectively, the port is useless here so we only extract the
+ip and assemble the url with jetty http port to get metrics and extract the value of the metric specified by metricName.
+
+See all available [server runtime metrics](https://github.com/apache/incubator-uniffle/blob/master/server/src/main/java/org/apache/uniffle/server/ShuffleServerMetrics.java).
+
+See all available [server grpc metrics](https://github.com/apache/incubator-uniffle/blob/master/server/src/main/java/org/apache/uniffle/server/ShuffleServerGrpcMetrics.java).
+
+See all available [coordinator runtime metrics](https://github.com/apache/incubator-uniffle/blob/master/coordinator/src/main/java/org/apache/uniffle/coordinator/CoordinatorMetrics.java).
+
+See all available [coordinator grpc metrics](https://github.com/apache/incubator-uniffle/blob/master/coordinator/src/main/java/org/apache/uniffle/coordinator/CoordinatorMetrics.java).
\ No newline at end of file