You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@fluo.apache.org by kt...@apache.org on 2016/09/23 21:33:48 UTC
incubator-fluo git commit: Added license header to docs
Repository: incubator-fluo
Updated Branches:
refs/heads/master 882299d77 -> 581b4dd50
Added license header to docs
Project: http://git-wip-us.apache.org/repos/asf/incubator-fluo/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-fluo/commit/581b4dd5
Tree: http://git-wip-us.apache.org/repos/asf/incubator-fluo/tree/581b4dd5
Diff: http://git-wip-us.apache.org/repos/asf/incubator-fluo/diff/581b4dd5
Branch: refs/heads/master
Commit: 581b4dd500faa3beca967bd13ad343c3256b9534
Parents: 882299d
Author: Keith Turner <kt...@apache.org>
Authored: Fri Sep 23 17:15:47 2016 -0400
Committer: Keith Turner <kt...@apache.org>
Committed: Fri Sep 23 17:15:47 2016 -0400
----------------------------------------------------------------------
README.md | 19 ++++++++++++++++-
docs/applications.md | 17 +++++++++++++++
docs/architecture.md | 19 ++++++++++++++++-
docs/contributing.md | 17 +++++++++++++++
docs/grafana.md | 17 +++++++++++++++
docs/install.md | 19 ++++++++++++++++-
docs/metrics.md | 53 +++++++++++++++++++++++++++++++----------------
pom.xml | 2 --
8 files changed, 140 insertions(+), 23 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/581b4dd5/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 35538c0..fb29e8f 100644
--- a/README.md
+++ b/README.md
@@ -1,3 +1,20 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
[![Fluo][logo]][fluo]
---
[![Build Status][ti]][tl] [![Apache License][li]][ll] [![Maven Central][mi]][ml] [![Javadoc][ji]][jl]
@@ -11,7 +28,7 @@ of a large-scale computation, index, or analytic as new data is discovered. Chec
## Getting Started
-* Take the [Fluo Tour][tour] if you are completely new to Fluo.
+* Take the [Fluo Tour][tour] if you are completely new to Fluo.
* Read the [install instructions][install] to install Fluo and start a Fluo application in YARN on a
cluster where Accumulo, Hadoop & Zookeeper are running. If you need help setting up these
dependencies, see the [related projects page][related] for external projects that may help.
http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/581b4dd5/docs/applications.md
----------------------------------------------------------------------
diff --git a/docs/applications.md b/docs/applications.md
index 2a1ed51..b7dd02b 100644
--- a/docs/applications.md
+++ b/docs/applications.md
@@ -1,3 +1,20 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
# Fluo Applications
Once you have Fluo installed and running on your cluster, you can now run Fluo applications which
http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/581b4dd5/docs/architecture.md
----------------------------------------------------------------------
diff --git a/docs/architecture.md b/docs/architecture.md
index cc36682..c353cba 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -1,3 +1,20 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
# Fluo Architecture
![fluo-architecture][1]
@@ -38,4 +55,4 @@ Fluo requires the following software to be running on the cluster:
* **Zookeeper** - Fluo stores its metadata and state information in Zookeeper. Zookeeper is also
required for Accumulo.
-[1]: resources/fluo-architecture.png
\ No newline at end of file
+[1]: resources/fluo-architecture.png
http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/581b4dd5/docs/contributing.md
----------------------------------------------------------------------
diff --git a/docs/contributing.md b/docs/contributing.md
index 5528ac4..0b43603 100644
--- a/docs/contributing.md
+++ b/docs/contributing.md
@@ -1,3 +1,20 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
# Contributing to Fluo
## Building Fluo
http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/581b4dd5/docs/grafana.md
----------------------------------------------------------------------
diff --git a/docs/grafana.md b/docs/grafana.md
index e7310f6..54f6397 100644
--- a/docs/grafana.md
+++ b/docs/grafana.md
@@ -1,3 +1,20 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
# Fluo metrics in Grafana/InfluxDB
This document describes how to send Fluo metrics to [InfluxDB], a time series database, and make
http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/581b4dd5/docs/install.md
----------------------------------------------------------------------
diff --git a/docs/install.md b/docs/install.md
index 76d3baa..18f4380 100644
--- a/docs/install.md
+++ b/docs/install.md
@@ -1,8 +1,25 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
# Fluo Install Instructions
Instructions for installing Apache Fluo and starting a Fluo application in YARN on a cluster where
Accumulo, Hadoop & Zookeeper are running. If you need help setting up these dependencies, see the
-[related projects page][related] for external projects that may help.
+[related projects page][related] for external projects that may help.
## Requirements
http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/581b4dd5/docs/metrics.md
----------------------------------------------------------------------
diff --git a/docs/metrics.md b/docs/metrics.md
index 35319e4..4240394 100644
--- a/docs/metrics.md
+++ b/docs/metrics.md
@@ -1,15 +1,32 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
# Fluo Metrics
-A Fluo application can be configured (in [fluo.properties]) to report metrics. When metrics are
+A Fluo application can be configured (in [fluo.properties]) to report metrics. When metrics are
configured, Fluo will report some 'default' metrics about an application that help users monitor its
-performance. Users can also write code to report 'application-specific' metrics from their
+performance. Users can also write code to report 'application-specific' metrics from their
applications. Both 'application-specific' and 'default' metrics share the same reporter configured
by [fluo.properties] and are described in detail below.
## Configuring reporters
Fluo metrics are not published by default. To publish metrics, configure a reporter in the 'metrics'
-section of [fluo.properties]. There are several different reporter types (i.e Console, CSV,
+section of [fluo.properties]. There are several different reporter types (i.e Console, CSV,
Graphite, JMX, SLF4J) that are implemented using [Dropwizard]. The choice of which reporter to use
depends on the visualization tool used. If you are not currently using a visualization tool, there
is [documentation][grafana] for reporting Fluo metrics to Grafana/InfluxDB.
@@ -24,15 +41,15 @@ Default metrics start with `fluo.class` or `fluo.system` and have following nami
fluo.class.APPLICATION.REPORTER_ID.METRIC.CLASS
fluo.system.APPLICATION.REPORTER_ID.METRIC
-
+
Application metrics start with `fluo.app` and have following scheme:
-
+
fluo.app.REPORTER_ID.METRIC
-
+
The variables below describe the additional information that is encoded in metrics names.
1. `APPLICATION` - Fluo application name
-2. `REPORTER_ID` - Unique ID of the Fluo oracle, worker, or client that is reporting the metric.
+2. `REPORTER_ID` - Unique ID of the Fluo oracle, worker, or client that is reporting the metric.
When running in YARN, this ID is of the format `worker-INSTANCE_ID` or `oracle-INSTANCE_ID`
where `INSTANCE_ID` corresponds to instance number. When not running in YARN, this ID consists
of a hostname and a base36 long that is unique across all fluo processes.
@@ -40,7 +57,7 @@ The variables below describe the additional information that is encoded in metri
metrics, this is set by user. Name should be unique and avoid using period '.' in name.
4. `CLASS` - Name of Fluo observer or loader class that produced metric. This allows things like
transaction collisions to be tracked per class.
-
+
## Application-specific metrics
Application metrics are implemented by retrieving a [MetricsReporter] from an [Observer], [Loader],
@@ -51,17 +68,17 @@ or [FluoClient]. These metrics are named using the format `fluo.app.REPORTER_ID
Default metrics report for a particular Observer/Loader class or system-wide.
Below are metrics that are reported from each Observer/Loader class that is configured in a Fluo
-application. These metrics are reported after each transaction and named using the format
+application. These metrics are reported after each transaction and named using the format
`fluo.class.APPLICATION.REPORTER_ID.METRIC.CLASS`.
-* tx_lock_wait_time - [Timer]
+* tx_lock_wait_time - [Timer]
- Time transaction spent waiting on locks held by other transactions.
- Only updated for transactions that have non-zero lock time.
* tx_execution_time - [Timer]
- - Time transaction took to execute.
+ - Time transaction took to execute.
- Updated for failed and successful transactions.
- This does not include commit time, only the time from start until commit is called.
-* tx_with_collision - [Meter]
+* tx_with_collision - [Meter]
- Rate of transactions with collisions.
* tx_collisions - [Meter]
- Rate of collisions.
@@ -69,15 +86,15 @@ application. These metrics are reported after each transaction and named using t
- Rate of row/columns set by transaction
* tx_entries_read - [Meter]
- Rate of row/columns read by transaction that existed.
- - There is currently no count of all reads (including non-existent data)
-* tx_locks_timedout - [Meter]
+ - There is currently no count of all reads (including non-existent data)
+* tx_locks_timedout - [Meter]
- Rate of timedout locks rolled back by transaction.
- These are locks that are held for very long periods by another transaction that appears to be
alive based on zookeeper.
-* tx_locks_dead - [Meter]
- - Rate of dead locks rolled by a transaction.
+* tx_locks_dead - [Meter]
+ - Rate of dead locks rolled by a transaction.
- These are locks held by a process that appears to be dead according to zookeeper.
-* tx_status_`<STATUS>` - [Meter]
+* tx_status_`<STATUS>` - [Meter]
- Rate of different ways (i.e `<STATUS>`) a transaction can terminate
Below are system-wide metrics that are reported for the entire Fluo application. These metrics are
@@ -91,7 +108,7 @@ named using the format `fluo.system.APPLICATION.REPORTER_ID.METRIC`.
- Number of stamps requested for each request for stamps from a client
* worker_notifications_queued - [Gauge]
- The current number of notifications queued for processing.
-* transactor_committing - [Gauge]
+* transactor_committing - [Gauge]
- The current number of transactions that are working their way through the commit steps.
Histograms and Timers have a counter. In the case of a histogram, the counter is the number of times
http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/581b4dd5/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index 17785c1..16e3870 100644
--- a/pom.xml
+++ b/pom.xml
@@ -273,8 +273,6 @@
<configuration>
<excludes>
<exclude>**/services/org.apache.fluo.core.metrics.ReporterStarter</exclude>
- <exclude>README.md</exclude>
- <exclude>docs/**.md</exclude>
<exclude>contrib/grafana/**.json</exclude>
</excludes>
</configuration>