You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sr...@apache.org on 2016/11/08 14:04:12 UTC
spark git commit: [MINOR][DOC] Unify example marks
Repository: spark
Updated Branches:
refs/heads/master ee2e741ac -> b1033fb74
[MINOR][DOC] Unify example marks
## What changes were proposed in this pull request?
1, `**Example**` => `**Examples**`, because more algos use `**Examples**`.
2, delete `### Examples` in `Isotonic regression`, because it's not that special in http://spark.apache.org/docs/latest/ml-classification-regression.html
3, add missing marks for `LDA` and other algos.
## How was this patch tested?
No tests for it only modify doc
Author: Zheng RuiFeng <ru...@foxmail.com>
Closes #15783 from zhengruifeng/doc_fix.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/b1033fb7
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/b1033fb7
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/b1033fb7
Branch: refs/heads/master
Commit: b1033fb74595716a8973acae43a6415d8e0a76d2
Parents: ee2e741
Author: Zheng RuiFeng <ru...@foxmail.com>
Authored: Tue Nov 8 14:04:07 2016 +0000
Committer: Sean Owen <so...@cloudera.com>
Committed: Tue Nov 8 14:04:07 2016 +0000
----------------------------------------------------------------------
docs/ml-classification-regression.md | 30 +++++++++++++++---------------
docs/ml-clustering.md | 8 +++++---
docs/ml-collaborative-filtering.md | 2 +-
docs/ml-features.md | 30 ++++++++++++++++++++++++++++++
docs/ml-tuning.md | 4 ++--
5 files changed, 53 insertions(+), 21 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/b1033fb7/docs/ml-classification-regression.md
----------------------------------------------------------------------
diff --git a/docs/ml-classification-regression.md b/docs/ml-classification-regression.md
index bb2e404..b10793d 100644
--- a/docs/ml-classification-regression.md
+++ b/docs/ml-classification-regression.md
@@ -46,7 +46,7 @@ parameter to select between these two algorithms, or leave it unset and Spark wi
For more background and more details about the implementation of binomial logistic regression, refer to the documentation of [logistic regression in `spark.mllib`](mllib-linear-methods.html#logistic-regression).
-**Example**
+**Examples**
The following example shows how to train binomial and multinomial logistic regression
models for binary classification with elastic net regularization. `elasticNetParam` corresponds to
@@ -137,7 +137,7 @@ We minimize the weighted negative log-likelihood, using a multinomial response m
For a detailed derivation please see [here](https://en.wikipedia.org/wiki/Multinomial_logistic_regression#As_a_log-linear_model).
-**Example**
+**Examples**
The following example shows how to train a multiclass logistic regression
model with elastic net regularization.
@@ -164,7 +164,7 @@ model with elastic net regularization.
Decision trees are a popular family of classification and regression methods.
More information about the `spark.ml` implementation can be found further in the [section on decision trees](#decision-trees).
-**Example**
+**Examples**
The following examples load a dataset in LibSVM format, split it into training and test sets, train on the first dataset, and then evaluate on the held-out test set.
We use two feature transformers to prepare the data; these help index categories for the label and categorical features, adding metadata to the `DataFrame` which the Decision Tree algorithm can recognize.
@@ -201,7 +201,7 @@ More details on parameters can be found in the [Python API documentation](api/py
Random forests are a popular family of classification and regression methods.
More information about the `spark.ml` implementation can be found further in the [section on random forests](#random-forests).
-**Example**
+**Examples**
The following examples load a dataset in LibSVM format, split it into training and test sets, train on the first dataset, and then evaluate on the held-out test set.
We use two feature transformers to prepare the data; these help index categories for the label and categorical features, adding metadata to the `DataFrame` which the tree-based algorithms can recognize.
@@ -234,7 +234,7 @@ Refer to the [Python API docs](api/python/pyspark.ml.html#pyspark.ml.classificat
Gradient-boosted trees (GBTs) are a popular classification and regression method using ensembles of decision trees.
More information about the `spark.ml` implementation can be found further in the [section on GBTs](#gradient-boosted-trees-gbts).
-**Example**
+**Examples**
The following examples load a dataset in LibSVM format, split it into training and test sets, train on the first dataset, and then evaluate on the held-out test set.
We use two feature transformers to prepare the data; these help index categories for the label and categorical features, adding metadata to the `DataFrame` which the tree-based algorithms can recognize.
@@ -284,7 +284,7 @@ The number of nodes `$N$` in the output layer corresponds to the number of class
MLPC employs backpropagation for learning the model. We use the logistic loss function for optimization and L-BFGS as an optimization routine.
-**Example**
+**Examples**
<div class="codetabs">
@@ -311,7 +311,7 @@ MLPC employs backpropagation for learning the model. We use the logistic loss fu
Predictions are done by evaluating each binary classifier and the index of the most confident classifier is output as label.
-**Example**
+**Examples**
The example below demonstrates how to load the
[Iris dataset](http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/multiclass/iris.scale), parse it as a DataFrame and perform multiclass classification using `OneVsRest`. The test error is calculated to measure the algorithm accuracy.
@@ -348,7 +348,7 @@ naive Bayes](http://nlp.stanford.edu/IR-book/html/htmledition/naive-bayes-text-c
and [Bernoulli naive Bayes](http://nlp.stanford.edu/IR-book/html/htmledition/the-bernoulli-model-1.html).
More information can be found in the section on [Naive Bayes in MLlib](mllib-naive-bayes.html#naive-bayes-sparkmllib).
-**Example**
+**Examples**
<div class="codetabs">
<div data-lang="scala" markdown="1">
@@ -383,7 +383,7 @@ summaries is similar to the logistic regression case.
> When fitting LinearRegressionModel without intercept on dataset with constant nonzero column by "l-bfgs" solver, Spark MLlib outputs zero coefficients for constant nonzero columns. This behavior is the same as R glmnet but different from LIBSVM.
-**Example**
+**Examples**
The following
example demonstrates training an elastic net regularized linear
@@ -511,7 +511,7 @@ others.
</tbody>
</table>
-**Example**
+**Examples**
The following example demonstrates training a GLM with a Gaussian response and identity link
function and extracting model summary statistics.
@@ -544,7 +544,7 @@ Refer to the [Python API docs](api/python/pyspark.ml.html#pyspark.ml.regression.
Decision trees are a popular family of classification and regression methods.
More information about the `spark.ml` implementation can be found further in the [section on decision trees](#decision-trees).
-**Example**
+**Examples**
The following examples load a dataset in LibSVM format, split it into training and test sets, train on the first dataset, and then evaluate on the held-out test set.
We use a feature transformer to index categorical features, adding metadata to the `DataFrame` which the Decision Tree algorithm can recognize.
@@ -579,7 +579,7 @@ More details on parameters can be found in the [Python API documentation](api/py
Random forests are a popular family of classification and regression methods.
More information about the `spark.ml` implementation can be found further in the [section on random forests](#random-forests).
-**Example**
+**Examples**
The following examples load a dataset in LibSVM format, split it into training and test sets, train on the first dataset, and then evaluate on the held-out test set.
We use a feature transformer to index categorical features, adding metadata to the `DataFrame` which the tree-based algorithms can recognize.
@@ -612,7 +612,7 @@ Refer to the [Python API docs](api/python/pyspark.ml.html#pyspark.ml.regression.
Gradient-boosted trees (GBTs) are a popular regression method using ensembles of decision trees.
More information about the `spark.ml` implementation can be found further in the [section on GBTs](#gradient-boosted-trees-gbts).
-**Example**
+**Examples**
Note: For this example dataset, `GBTRegressor` actually only needs 1 iteration, but that will not
be true in general.
@@ -700,7 +700,7 @@ The implementation matches the result from R's survival function
> When fitting AFTSurvivalRegressionModel without intercept on dataset with constant nonzero column, Spark MLlib outputs zero coefficients for constant nonzero columns. This behavior is different from R survival::survreg.
-**Example**
+**Examples**
<div class="codetabs">
@@ -765,7 +765,7 @@ is treated as piecewise linear function. The rules for prediction therefore are:
predictions of the two closest features. In case there are multiple values
with the same feature then the same rules as in previous point are used.
-### Examples
+**Examples**
<div class="codetabs">
<div data-lang="scala" markdown="1">
http://git-wip-us.apache.org/repos/asf/spark/blob/b1033fb7/docs/ml-clustering.md
----------------------------------------------------------------------
diff --git a/docs/ml-clustering.md b/docs/ml-clustering.md
index 8a0a61c..eedacb1 100644
--- a/docs/ml-clustering.md
+++ b/docs/ml-clustering.md
@@ -65,7 +65,7 @@ called [kmeans||](http://theory.stanford.edu/~sergei/papers/vldb12-kmpar.pdf).
</tbody>
</table>
-### Example
+**Examples**
<div class="codetabs">
@@ -94,6 +94,8 @@ Refer to the [Python API docs](api/python/pyspark.ml.html#pyspark.ml.clustering.
and generates a `LDAModel` as the base model. Expert users may cast a `LDAModel` generated by
`EMLDAOptimizer` to a `DistributedLDAModel` if needed.
+**Examples**
+
<div class="codetabs">
<div data-lang="scala" markdown="1">
@@ -128,7 +130,7 @@ Bisecting K-means can often be much faster than regular K-means, but it will gen
`BisectingKMeans` is implemented as an `Estimator` and generates a `BisectingKMeansModel` as the base model.
-### Example
+**Examples**
<div class="codetabs">
@@ -210,7 +212,7 @@ model.
</tbody>
</table>
-### Example
+**Examples**
<div class="codetabs">
http://git-wip-us.apache.org/repos/asf/spark/blob/b1033fb7/docs/ml-collaborative-filtering.md
----------------------------------------------------------------------
diff --git a/docs/ml-collaborative-filtering.md b/docs/ml-collaborative-filtering.md
index 1d02d69..4d19b40 100644
--- a/docs/ml-collaborative-filtering.md
+++ b/docs/ml-collaborative-filtering.md
@@ -59,7 +59,7 @@ This approach is named "ALS-WR" and discussed in the paper
It makes `regParam` less dependent on the scale of the dataset, so we can apply the
best parameter learned from a sampled subset to the full dataset and expect similar performance.
-## Examples
+**Examples**
<div class="codetabs">
<div data-lang="scala" markdown="1">
http://git-wip-us.apache.org/repos/asf/spark/blob/b1033fb7/docs/ml-features.md
----------------------------------------------------------------------
diff --git a/docs/ml-features.md b/docs/ml-features.md
index 9031772..19ec574 100644
--- a/docs/ml-features.md
+++ b/docs/ml-features.md
@@ -112,6 +112,8 @@ can then be used as features for prediction, document similarity calculations, e
Please refer to the [MLlib user guide on Word2Vec](mllib-feature-extraction.html#word2vec) for more
details.
+**Examples**
+
In the following code segment, we start with a set of documents, each of which is represented as a sequence of words. For each document, we transform it into a feature vector. This feature vector could then be passed to a learning algorithm.
<div class="codetabs">
@@ -220,6 +222,8 @@ for more details on the API.
Alternatively, users can set parameter "gaps" to false indicating the regex "pattern" denotes
"tokens" rather than splitting gaps, and find all matching occurrences as the tokenization result.
+**Examples**
+
<div class="codetabs">
<div data-lang="scala" markdown="1">
@@ -321,6 +325,8 @@ An [n-gram](https://en.wikipedia.org/wiki/N-gram) is a sequence of $n$ tokens (t
`NGram` takes as input a sequence of strings (e.g. the output of a [Tokenizer](ml-features.html#tokenizer)). The parameter `n` is used to determine the number of terms in each $n$-gram. The output will consist of a sequence of $n$-grams where each $n$-gram is represented by a space-delimited string of $n$ consecutive words. If the input sequence contains fewer than `n` strings, no output is produced.
+**Examples**
+
<div class="codetabs">
<div data-lang="scala" markdown="1">
@@ -358,6 +364,8 @@ for binarization. Feature values greater than the threshold are binarized to 1.0
to or less than the threshold are binarized to 0.0. Both Vector and Double types are supported
for `inputCol`.
+**Examples**
+
<div class="codetabs">
<div data-lang="scala" markdown="1">
@@ -388,6 +396,8 @@ for more details on the API.
[PCA](http://en.wikipedia.org/wiki/Principal_component_analysis) is a statistical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components. A [PCA](api/scala/index.html#org.apache.spark.ml.feature.PCA) class trains a model to project vectors to a low-dimensional space using PCA. The example below shows how to project 5-dimensional feature vectors into 3-dimensional principal components.
+**Examples**
+
<div class="codetabs">
<div data-lang="scala" markdown="1">
@@ -418,6 +428,8 @@ for more details on the API.
[Polynomial expansion](http://en.wikipedia.org/wiki/Polynomial_expansion) is the process of expanding your features into a polynomial space, which is formulated by an n-degree combination of original dimensions. A [PolynomialExpansion](api/scala/index.html#org.apache.spark.ml.feature.PolynomialExpansion) class provides this functionality. The example below shows how to expand your features into a 3-degree polynomial space.
+**Examples**
+
<div class="codetabs">
<div data-lang="scala" markdown="1">
@@ -458,6 +470,8 @@ for the transform is unitary. No shift is applied to the transformed
sequence (e.g. the $0$th element of the transformed sequence is the
$0$th DCT coefficient and _not_ the $N/2$th).
+**Examples**
+
<div class="codetabs">
<div data-lang="scala" markdown="1">
@@ -663,6 +677,8 @@ for more details on the API.
[One-hot encoding](http://en.wikipedia.org/wiki/One-hot) maps a column of label indices to a column of binary vectors, with at most a single one-value. This encoding allows algorithms which expect continuous features, such as Logistic Regression, to use categorical features.
+**Examples**
+
<div class="codetabs">
<div data-lang="scala" markdown="1">
@@ -701,6 +717,8 @@ It can both automatically decide which features are categorical and convert orig
Indexing categorical features allows algorithms such as Decision Trees and Tree Ensembles to treat categorical features appropriately, improving performance.
+**Examples**
+
In the example below, we read in a dataset of labeled points and then use `VectorIndexer` to decide which features should be treated as categorical. We transform the categorical feature values to their indices. This transformed data could then be passed to algorithms such as `DecisionTreeRegressor` that handle categorical features.
<div class="codetabs">
@@ -786,6 +804,8 @@ for more details on the API.
`Normalizer` is a `Transformer` which transforms a dataset of `Vector` rows, normalizing each `Vector` to have unit norm. It takes parameter `p`, which specifies the [p-norm](http://en.wikipedia.org/wiki/Norm_%28mathematics%29#p-norm) used for normalization. ($p = 2$ by default.) This normalization can help standardize your input data and improve the behavior of learning algorithms.
+**Examples**
+
The following example demonstrates how to load a dataset in libsvm format and then normalize each row to have unit $L^1$ norm and unit $L^\infty$ norm.
<div class="codetabs">
@@ -826,6 +846,8 @@ for more details on the API.
Note that if the standard deviation of a feature is zero, it will return default `0.0` value in the `Vector` for that feature.
+**Examples**
+
The following example demonstrates how to load a dataset in libsvm format and then normalize each feature to have unit standard deviation.
<div class="codetabs">
@@ -871,6 +893,8 @@ For the case `$E_{max} == E_{min}$`, `$Rescaled(e_i) = 0.5 * (max + min)$`
Note that since zero values will probably be transformed to non-zero values, output of the transformer will be `DenseVector` even for sparse input.
+**Examples**
+
The following example demonstrates how to load a dataset in libsvm format and then rescale each feature to [0, 1].
<div class="codetabs">
@@ -912,6 +936,8 @@ data, and thus does not destroy any sparsity.
`MaxAbsScaler` computes summary statistics on a data set and produces a `MaxAbsScalerModel`. The
model can then transform each feature individually to range [-1, 1].
+**Examples**
+
The following example demonstrates how to load a dataset in libsvm format and then rescale each feature to [-1, 1].
<div class="codetabs">
@@ -955,6 +981,8 @@ Note also that the splits that you provided have to be in strictly increasing or
More details can be found in the API docs for [Bucketizer](api/scala/index.html#org.apache.spark.ml.feature.Bucketizer).
+**Examples**
+
The following example demonstrates how to bucketize a column of `Double`s into another index-wised column.
<div class="codetabs">
@@ -1003,6 +1031,8 @@ v_N
\end{pmatrix}
\]`
+**Examples**
+
This example below demonstrates how to transform vectors using a transforming vector value.
<div class="codetabs">
http://git-wip-us.apache.org/repos/asf/spark/blob/b1033fb7/docs/ml-tuning.md
----------------------------------------------------------------------
diff --git a/docs/ml-tuning.md b/docs/ml-tuning.md
index 2ca90c7..e4b0703 100644
--- a/docs/ml-tuning.md
+++ b/docs/ml-tuning.md
@@ -62,7 +62,7 @@ To help construct the parameter grid, users can use the [`ParamGridBuilder`](api
After identifying the best `ParamMap`, `CrossValidator` finally re-fits the `Estimator` using the best `ParamMap` and the entire dataset.
-## Example: model selection via cross-validation
+**Examples: model selection via cross-validation**
The following example demonstrates using `CrossValidator` to select from a grid of parameters.
@@ -102,7 +102,7 @@ It splits the dataset into these two parts using the `trainRatio` parameter. For
Like `CrossValidator`, `TrainValidationSplit` finally fits the `Estimator` using the best `ParamMap` and the entire dataset.
-## Example: model selection via train validation split
+**Examples: model selection via train validation split**
<div class="codetabs">
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org