You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2018/12/05 11:30:32 UTC
spark git commit: [SPARK-26133][ML][FOLLOWUP] Fix doc for
OneHotEncoder
Repository: spark
Updated Branches:
refs/heads/master 7e3eb3cd2 -> 169d9ad8f
[SPARK-26133][ML][FOLLOWUP] Fix doc for OneHotEncoder
## What changes were proposed in this pull request?
This fixes doc of renamed OneHotEncoder in PySpark.
## How was this patch tested?
N/A
Closes #23230 from viirya/remove_one_hot_encoder_followup.
Authored-by: Liang-Chi Hsieh <vi...@gmail.com>
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/169d9ad8
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/169d9ad8
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/169d9ad8
Branch: refs/heads/master
Commit: 169d9ad8f1b6006c8db0edbdfffc20dc73c78610
Parents: 7e3eb3c
Author: Liang-Chi Hsieh <vi...@gmail.com>
Authored: Wed Dec 5 19:30:25 2018 +0800
Committer: Hyukjin Kwon <gu...@apache.org>
Committed: Wed Dec 5 19:30:25 2018 +0800
----------------------------------------------------------------------
python/pyspark/ml/feature.py | 22 +++++++++++-----------
1 file changed, 11 insertions(+), 11 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/169d9ad8/python/pyspark/ml/feature.py
----------------------------------------------------------------------
diff --git a/python/pyspark/ml/feature.py b/python/pyspark/ml/feature.py
index 6cc80e1..c9507c2 100755
--- a/python/pyspark/ml/feature.py
+++ b/python/pyspark/ml/feature.py
@@ -1648,22 +1648,22 @@ class OneHotEncoder(JavaEstimator, HasInputCols, HasOutputCols, HasHandleInvalid
at most a single one-value per row that indicates the input category index.
For example with 5 categories, an input value of 2.0 would map to an output vector of
`[0.0, 0.0, 1.0, 0.0]`.
- The last category is not included by default (configurable via `dropLast`),
+ The last category is not included by default (configurable via :py:attr:`dropLast`),
because it makes the vector entries sum up to one, and hence linearly dependent.
So an input value of 4.0 maps to `[0.0, 0.0, 0.0, 0.0]`.
- Note: This is different from scikit-learn's OneHotEncoder, which keeps all categories.
- The output vectors are sparse.
+ .. note:: This is different from scikit-learn's OneHotEncoder, which keeps all categories.
+ The output vectors are sparse.
- When `handleInvalid` is configured to 'keep', an extra "category" indicating invalid values is
- added as last category. So when `dropLast` is true, invalid values are encoded as all-zeros
- vector.
+ When :py:attr:`handleInvalid` is configured to 'keep', an extra "category" indicating invalid
+ values is added as last category. So when :py:attr:`dropLast` is true, invalid values are
+ encoded as all-zeros vector.
- Note: When encoding multi-column by using `inputCols` and `outputCols` params, input/output
- cols come in pairs, specified by the order in the arrays, and each pair is treated
- independently.
+ .. note:: When encoding multi-column by using :py:attr:`inputCols` and
+ :py:attr:`outputCols` params, input/output cols come in pairs, specified by the order in
+ the arrays, and each pair is treated independently.
- See `StringIndexer` for converting categorical values into category indices
+ .. seealso:: :py:class:`StringIndexer` for converting categorical values into category indices
>>> from pyspark.ml.linalg import Vectors
>>> df = spark.createDataFrame([(0.0,), (1.0,), (2.0,)], ["input"])
@@ -1671,7 +1671,7 @@ class OneHotEncoder(JavaEstimator, HasInputCols, HasOutputCols, HasHandleInvalid
>>> model = ohe.fit(df)
>>> model.transform(df).head().output
SparseVector(2, {0: 1.0})
- >>> ohePath = temp_path + "/oheEstimator"
+ >>> ohePath = temp_path + "/ohe"
>>> ohe.save(ohePath)
>>> loadedOHE = OneHotEncoder.load(ohePath)
>>> loadedOHE.getInputCols() == ohe.getInputCols()
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org