You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by we...@apache.org on 2019/05/09 06:21:31 UTC
[spark-website] branch asf-site updated: add pyspark apis (#202)
This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/spark-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 381b3ca add pyspark apis (#202)
381b3ca is described below
commit 381b3ca29f19b4df74aacd99e96cd39f00ceb668
Author: Xiao Li <ga...@gmail.com>
AuthorDate: Wed May 8 23:21:26 2019 -0700
add pyspark apis (#202)
---
site/docs/2.4.3/api/python/_modules/index.html | 35 +
.../api/python/_modules/pyspark/accumulators.html | 1 +
.../api/python/_modules/pyspark/broadcast.html | 1 +
.../2.4.3/api/python/_modules/pyspark/conf.html | 1 +
.../2.4.3/api/python/_modules/pyspark/context.html | 1 +
.../2.4.3/api/python/_modules/pyspark/files.html | 1 +
.../2.4.3/api/python/_modules/pyspark/ml/base.html | 32 +-
.../python/_modules/pyspark/ml/classification.html | 32 +-
.../api/python/_modules/pyspark/ml/clustering.html | 38 +-
.../api/python/_modules/pyspark/ml/evaluation.html | 32 +-
.../api/python/_modules/pyspark/ml/feature.html | 43 +-
.../2.4.3/api/python/_modules/pyspark/ml/fpm.html | 32 +-
.../api/python/_modules/pyspark/ml/image.html | 39 +-
.../api/python/_modules/pyspark/ml/linalg.html | 38 +-
.../api/python/_modules/pyspark/ml/param.html | 32 +-
.../python/_modules/pyspark/ml/param/shared.html | 32 +-
.../api/python/_modules/pyspark/ml/pipeline.html | 32 +-
.../python/_modules/pyspark/ml/recommendation.html | 32 +-
.../api/python/_modules/pyspark/ml/regression.html | 32 +-
.../2.4.3/api/python/_modules/pyspark/ml/stat.html | 34 +-
.../api/python/_modules/pyspark/ml/tuning.html | 32 +-
.../2.4.3/api/python/_modules/pyspark/ml/util.html | 32 +-
.../api/python/_modules/pyspark/ml/wrapper.html | 32 +-
.../_modules/pyspark/mllib/classification.html | 32 +-
.../python/_modules/pyspark/mllib/clustering.html | 32 +-
.../api/python/_modules/pyspark/mllib/common.html | 32 +-
.../python/_modules/pyspark/mllib/evaluation.html | 32 +-
.../api/python/_modules/pyspark/mllib/feature.html | 32 +-
.../api/python/_modules/pyspark/mllib/fpm.html | 32 +-
.../api/python/_modules/pyspark/mllib/linalg.html | 38 +-
.../_modules/pyspark/mllib/linalg/distributed.html | 32 +-
.../api/python/_modules/pyspark/mllib/random.html | 32 +-
.../_modules/pyspark/mllib/recommendation.html | 32 +-
.../python/_modules/pyspark/mllib/regression.html | 32 +-
.../_modules/pyspark/mllib/stat/KernelDensity.html | 32 +-
.../_modules/pyspark/mllib/stat/distribution.html | 32 +-
.../python/_modules/pyspark/mllib/stat/test.html | 32 +-
.../api/python/_modules/pyspark/mllib/tree.html | 32 +-
.../api/python/_modules/pyspark/mllib/util.html | 32 +-
.../api/python/_modules/pyspark/profiler.html | 1 +
.../2.4.3/api/python/_modules/pyspark/rdd.html | 1 +
.../api/python/_modules/pyspark/serializers.html | 1 +
.../api/python/_modules/pyspark/sql/catalog.html | 1 +
.../api/python/_modules/pyspark/sql/column.html | 1 +
.../api/python/_modules/pyspark/sql/context.html | 1 +
.../api/python/_modules/pyspark/sql/dataframe.html | 1 +
.../api/python/_modules/pyspark/sql/functions.html | 1 +
.../api/python/_modules/pyspark/sql/group.html | 1 +
.../python/_modules/pyspark/sql/readwriter.html | 1 +
.../api/python/_modules/pyspark/sql/session.html | 1 +
.../api/python/_modules/pyspark/sql/streaming.html | 1 +
.../api/python/_modules/pyspark/sql/types.html | 1 +
.../2.4.3/api/python/_modules/pyspark/sql/udf.html | 1 +
.../api/python/_modules/pyspark/sql/window.html | 1 +
.../2.4.3/api/python/_modules/pyspark/status.html | 1 +
.../api/python/_modules/pyspark/storagelevel.html | 1 +
.../python/_modules/pyspark/streaming/context.html | 1 +
.../python/_modules/pyspark/streaming/dstream.html | 1 +
.../python/_modules/pyspark/streaming/flume.html | 1 +
.../python/_modules/pyspark/streaming/kafka.html | 1 +
.../python/_modules/pyspark/streaming/kinesis.html | 1 +
.../_modules/pyspark/streaming/listener.html | 1 +
.../api/python/_modules/pyspark/taskcontext.html | 1 +
site/docs/2.4.3/api/python/index.html | 55 +-
site/docs/2.4.3/api/python/objects.inv | Bin 5617 -> 27022 bytes
site/docs/2.4.3/api/python/pyspark.html | 1 +
site/docs/2.4.3/api/python/pyspark.ml.html | 36005 ++++++++++++++++++-
site/docs/2.4.3/api/python/pyspark.mllib.html | 7981 +++-
site/docs/2.4.3/api/python/search.html | 1 +
site/docs/2.4.3/api/python/searchindex.js | 2 +-
70 files changed, 44391 insertions(+), 812 deletions(-)
diff --git a/site/docs/2.4.3/api/python/_modules/index.html b/site/docs/2.4.3/api/python/_modules/index.html
index aef3987..b38ce2e 100644
--- a/site/docs/2.4.3/api/python/_modules/index.html
+++ b/site/docs/2.4.3/api/python/_modules/index.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<script type="text/javascript" src="../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../search.html" />
</head><body>
@@ -37,6 +38,40 @@
<li><a href="pyspark/conf.html">pyspark.conf</a></li>
<li><a href="pyspark/context.html">pyspark.context</a></li>
<li><a href="pyspark/files.html">pyspark.files</a></li>
+<li><a href="pyspark/ml/base.html">pyspark.ml.base</a></li>
+<li><a href="pyspark/ml/classification.html">pyspark.ml.classification</a></li>
+<li><a href="pyspark/ml/clustering.html">pyspark.ml.clustering</a></li>
+<li><a href="pyspark/ml/evaluation.html">pyspark.ml.evaluation</a></li>
+<li><a href="pyspark/ml/feature.html">pyspark.ml.feature</a></li>
+<li><a href="pyspark/ml/fpm.html">pyspark.ml.fpm</a></li>
+<li><a href="pyspark/ml/image.html">pyspark.ml.image</a></li>
+<li><a href="pyspark/ml/linalg.html">pyspark.ml.linalg</a></li>
+<li><a href="pyspark/ml/param.html">pyspark.ml.param</a></li>
+<ul><li><a href="pyspark/ml/param/shared.html">pyspark.ml.param.shared</a></li>
+</ul><li><a href="pyspark/ml/pipeline.html">pyspark.ml.pipeline</a></li>
+<li><a href="pyspark/ml/recommendation.html">pyspark.ml.recommendation</a></li>
+<li><a href="pyspark/ml/regression.html">pyspark.ml.regression</a></li>
+<li><a href="pyspark/ml/stat.html">pyspark.ml.stat</a></li>
+<li><a href="pyspark/ml/tuning.html">pyspark.ml.tuning</a></li>
+<li><a href="pyspark/ml/util.html">pyspark.ml.util</a></li>
+<li><a href="pyspark/ml/wrapper.html">pyspark.ml.wrapper</a></li>
+<li><a href="pyspark/mllib/classification.html">pyspark.mllib.classification</a></li>
+<li><a href="pyspark/mllib/clustering.html">pyspark.mllib.clustering</a></li>
+<li><a href="pyspark/mllib/common.html">pyspark.mllib.common</a></li>
+<li><a href="pyspark/mllib/evaluation.html">pyspark.mllib.evaluation</a></li>
+<li><a href="pyspark/mllib/feature.html">pyspark.mllib.feature</a></li>
+<li><a href="pyspark/mllib/fpm.html">pyspark.mllib.fpm</a></li>
+<li><a href="pyspark/mllib/linalg.html">pyspark.mllib.linalg</a></li>
+<ul><li><a href="pyspark/mllib/linalg/distributed.html">pyspark.mllib.linalg.distributed</a></li>
+</ul><li><a href="pyspark/mllib/random.html">pyspark.mllib.random</a></li>
+<li><a href="pyspark/mllib/recommendation.html">pyspark.mllib.recommendation</a></li>
+<li><a href="pyspark/mllib/regression.html">pyspark.mllib.regression</a></li>
+<li><a href="pyspark/mllib/stat/KernelDensity.html">pyspark.mllib.stat.KernelDensity</a></li>
+<li><a href="pyspark/mllib/stat/_statistics.html">pyspark.mllib.stat._statistics</a></li>
+<li><a href="pyspark/mllib/stat/distribution.html">pyspark.mllib.stat.distribution</a></li>
+<li><a href="pyspark/mllib/stat/test.html">pyspark.mllib.stat.test</a></li>
+<li><a href="pyspark/mllib/tree.html">pyspark.mllib.tree</a></li>
+<li><a href="pyspark/mllib/util.html">pyspark.mllib.util</a></li>
<li><a href="pyspark/profiler.html">pyspark.profiler</a></li>
<li><a href="pyspark/rdd.html">pyspark.rdd</a></li>
<li><a href="pyspark/serializers.html">pyspark.serializers</a></li>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/accumulators.html b/site/docs/2.4.3/api/python/_modules/pyspark/accumulators.html
index 2a45eb4..e4da200 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/accumulators.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/accumulators.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../_static/underscore.js"></script>
<script type="text/javascript" src="../../_static/doctools.js"></script>
<script type="text/javascript" src="../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/broadcast.html b/site/docs/2.4.3/api/python/_modules/pyspark/broadcast.html
index 6c85af3..8764ece 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/broadcast.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/broadcast.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../_static/underscore.js"></script>
<script type="text/javascript" src="../../_static/doctools.js"></script>
<script type="text/javascript" src="../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/conf.html b/site/docs/2.4.3/api/python/_modules/pyspark/conf.html
index 0ac68fc..782242d 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/conf.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/conf.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../_static/underscore.js"></script>
<script type="text/javascript" src="../../_static/doctools.js"></script>
<script type="text/javascript" src="../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/context.html b/site/docs/2.4.3/api/python/_modules/pyspark/context.html
index d0803fa..4f3598b 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/context.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/context.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../_static/underscore.js"></script>
<script type="text/javascript" src="../../_static/doctools.js"></script>
<script type="text/javascript" src="../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/files.html b/site/docs/2.4.3/api/python/_modules/pyspark/files.html
index c8ba784..c93d37e 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/files.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/files.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../_static/underscore.js"></script>
<script type="text/javascript" src="../../_static/doctools.js"></script>
<script type="text/javascript" src="../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/ml/base.html b/site/docs/2.4.3/api/python/_modules/pyspark/ml/base.html
index fa6ca48..436321b 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/ml/base.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/ml/base.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.ml.base — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -292,12 +282,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -315,7 +305,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/ml/classification.html b/site/docs/2.4.3/api/python/_modules/pyspark/ml/classification.html
index d3f85d3..429dedc 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/ml/classification.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/ml/classification.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.ml.classification — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -2145,12 +2135,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -2168,7 +2158,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/ml/clustering.html b/site/docs/2.4.3/api/python/_modules/pyspark/ml/clustering.html
index 390980e..2c1d8f7 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/ml/clustering.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/ml/clustering.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.ml.clustering — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -585,13 +575,7 @@
<span class="sd">"""</span>
<span class="sd"> Computes the sum of squared distances between the input points</span>
<span class="sd"> and their corresponding cluster centers.</span>
-
-<span class="sd"> ..note:: Deprecated in 2.4.0. It will be removed in 3.0.0. Use ClusteringEvaluator instead.</span>
-<span class="sd"> You can also get the cost on the training dataset in the summary.</span>
<span class="sd"> """</span>
- <span class="n">warnings</span><span class="o">.</span><span class="n">warn</span><span class="p">(</span><span class="s2">"Deprecated in 2.4.0. It will be removed in 3.0.0. Use ClusteringEvaluator "</span>
- <span class="s2">"instead. You can also get the cost on the training dataset in the summary."</span><span class="p">,</span>
- <span class="ne">DeprecationWarning</span><span class="p">)</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_call_java</span><span class="p">(</span><span class="s2">"computeCost"</span><span class="p">,</span> <span class="n">dataset</span><span class="p">)</span></div>
<span class="nd">@property</span>
@@ -1465,12 +1449,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -1488,7 +1472,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/ml/evaluation.html b/site/docs/2.4.3/api/python/_modules/pyspark/ml/evaluation.html
index 0a91828..98f1410 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/ml/evaluation.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/ml/evaluation.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.ml.evaluation — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -505,12 +495,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -528,7 +518,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/ml/feature.html b/site/docs/2.4.3/api/python/_modules/pyspark/ml/feature.html
index 3194780..cf9ea3b 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/ml/feature.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/ml/feature.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.ml.feature — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -238,6 +228,7 @@
<span class="sd"> "datasetA" and "datasetB", and a column "distCol" is added to show the distance</span>
<span class="sd"> between each pair.</span>
<span class="sd"> """</span>
+ <span class="n">threshold</span> <span class="o">=</span> <span class="n">TypeConverters</span><span class="o">.</span><span class="n">toFloat</span><span class="p">(</span><span class="n">threshold</span><span class="p">)</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_call_java</span><span class="p">(</span><span class="s2">"approxSimilarityJoin"</span><span class="p">,</span> <span class="n">datasetA</span><span class="p">,</span> <span class="n">datasetB</span><span class="p">,</span> <span class="n">threshold</span><span class="p">,</span> <span class="n">distCol</span><span class="p">)</span>
@@ -285,6 +276,16 @@
<span class="sd"> | 3| 6| 2.23606797749979|</span>
<span class="sd"> +---+---+-----------------+</span>
<span class="sd"> ...</span>
+<span class="sd"> >>> model.approxSimilarityJoin(df, df2, 3, distCol="EuclideanDistance").select(</span>
+<span class="sd"> ... col("datasetA.id").alias("idA"),</span>
+<span class="sd"> ... col("datasetB.id").alias("idB"),</span>
+<span class="sd"> ... col("EuclideanDistance")).show()</span>
+<span class="sd"> +---+---+-----------------+</span>
+<span class="sd"> |idA|idB|EuclideanDistance|</span>
+<span class="sd"> +---+---+-----------------+</span>
+<span class="sd"> | 3| 6| 2.23606797749979|</span>
+<span class="sd"> +---+---+-----------------+</span>
+<span class="sd"> ...</span>
<span class="sd"> >>> brpPath = temp_path + "/brp"</span>
<span class="sd"> >>> brp.save(brpPath)</span>
<span class="sd"> >>> brp2 = BucketedRandomProjectionLSH.load(brpPath)</span>
@@ -3945,12 +3946,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -3968,7 +3969,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/ml/fpm.html b/site/docs/2.4.3/api/python/_modules/pyspark/ml/fpm.html
index 16b80e8..7cb7487 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/ml/fpm.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/ml/fpm.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.ml.fpm — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -404,12 +394,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -427,7 +417,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/ml/image.html b/site/docs/2.4.3/api/python/_modules/pyspark/ml/image.html
index 2717949..a077a4b 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/ml/image.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/ml/image.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.ml.image — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -73,6 +63,7 @@
<span class="kn">import</span> <span class="nn">warnings</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
+<span class="kn">from</span> <span class="nn">distutils.version</span> <span class="k">import</span> <span class="n">LooseVersion</span>
<span class="kn">from</span> <span class="nn">pyspark</span> <span class="k">import</span> <span class="n">SparkContext</span>
<span class="kn">from</span> <span class="nn">pyspark.sql.types</span> <span class="k">import</span> <span class="n">Row</span><span class="p">,</span> <span class="n">_create_row</span><span class="p">,</span> <span class="n">_parse_datatype_json_string</span>
@@ -235,7 +226,11 @@
<span class="c1"># Running `bytearray(numpy.array([1]))` fails in specific Python versions</span>
<span class="c1"># with a specific Numpy version, for example in Python 3.6.0 and NumPy 1.13.3.</span>
<span class="c1"># Here, it avoids it by converting it to bytes.</span>
- <span class="n">data</span> <span class="o">=</span> <span class="nb">bytearray</span><span class="p">(</span><span class="n">array</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">dtype</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">uint8</span><span class="p">)</span><span class="o">.</span><span class="n">ravel</span><span class="p">()</span><span class="o">.</span><span class="n" [...]
+ <span class="k">if</span> <span class="n">LooseVersion</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">__version__</span><span class="p">)</span> <span class="o">>=</span> <span class="n">LooseVersion</span><span class="p">(</span><span class="s1">'1.9'</span><span class="p">):</span>
+ <span class="n">data</span> <span class="o">=</span> <span class="nb">bytearray</span><span class="p">(</span><span class="n">array</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">dtype</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">uint8</span><span class="p">)</span><span class="o">.</span><span class="n">ravel</span><span class="p">()</span><span class="o">.</span><span class [...]
+ <span class="k">else</span><span class="p">:</span>
+ <span class="c1"># Numpy prior to 1.9 don't have `tobytes` method.</span>
+ <span class="n">data</span> <span class="o">=</span> <span class="nb">bytearray</span><span class="p">(</span><span class="n">array</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">dtype</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">uint8</span><span class="p">)</span><span class="o">.</span><span class="n">ravel</span><span class="p">())</span>
<span class="c1"># Creating new Row with _create_row(), because Row(name = value, ... )</span>
<span class="c1"># orders fields by name, which conflicts with expected schema order</span>
@@ -323,12 +318,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -346,7 +341,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/ml/linalg.html b/site/docs/2.4.3/api/python/_modules/pyspark/ml/linalg.html
index 9ac38fd..faa9388 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/ml/linalg.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/ml/linalg.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.ml.linalg — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -315,6 +305,8 @@
<span class="sd"> DenseVector([3.0, 2.0])</span>
<span class="sd"> >>> u % 2</span>
<span class="sd"> DenseVector([1.0, 0.0])</span>
+<span class="sd"> >>> -v</span>
+<span class="sd"> DenseVector([-1.0, -2.0])</span>
<span class="sd"> """</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">ar</span><span class="p">):</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">ar</span><span class="p">,</span> <span class="nb">bytes</span><span class="p">):</span>
@@ -481,6 +473,9 @@
<span class="k">def</span> <span class="nf">__getattr__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">item</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">getattr</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">array</span><span class="p">,</span> <span class="n">item</span><span class="p">)</span>
+ <span class="k">def</span> <span class="nf">__neg__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+ <span class="k">return</span> <span class="n">DenseVector</span><span class="p">(</span><span class="o">-</span><span class="bp">self</span><span class="o">.</span><span class="n">array</span><span class="p">)</span>
+
<span class="k">def</span> <span class="nf">_delegate</span><span class="p">(</span><span class="n">op</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">func</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">other</span><span class="p">):</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">other</span><span class="p">,</span> <span class="n">DenseVector</span><span class="p">):</span>
@@ -488,7 +483,6 @@
<span class="k">return</span> <span class="n">DenseVector</span><span class="p">(</span><span class="nb">getattr</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">array</span><span class="p">,</span> <span class="n">op</span><span class="p">)(</span><span class="n">other</span><span class="p">))</span>
<span class="k">return</span> <span class="n">func</span>
- <span class="fm">__neg__</span> <span class="o">=</span> <span class="n">_delegate</span><span class="p">(</span><span class="s2">"__neg__"</span><span class="p">)</span>
<span class="fm">__add__</span> <span class="o">=</span> <span class="n">_delegate</span><span class="p">(</span><span class="s2">"__add__"</span><span class="p">)</span>
<span class="fm">__sub__</span> <span class="o">=</span> <span class="n">_delegate</span><span class="p">(</span><span class="s2">"__sub__"</span><span class="p">)</span>
<span class="fm">__mul__</span> <span class="o">=</span> <span class="n">_delegate</span><span class="p">(</span><span class="s2">"__mul__"</span><span class="p">)</span>
@@ -1224,12 +1218,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -1247,7 +1241,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/ml/param.html b/site/docs/2.4.3/api/python/_modules/pyspark/ml/param.html
index d5de527..7ef1ee9 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/ml/param.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/ml/param.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.ml.param — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -566,12 +556,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -589,7 +579,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/ml/param/shared.html b/site/docs/2.4.3/api/python/_modules/pyspark/ml/param/shared.html
index d64b602..e9c019e 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/ml/param/shared.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/ml/param/shared.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.ml.param.shared — PySpark master documentation</title>
<link rel="stylesheet" href="../../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../../" src="../../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -872,12 +862,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -896,7 +886,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/ml/pipeline.html b/site/docs/2.4.3/api/python/_modules/pyspark/ml/pipeline.html
index 313cc4c..3044158 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/ml/pipeline.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/ml/pipeline.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.ml.pipeline — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -445,12 +435,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -468,7 +458,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/ml/recommendation.html b/site/docs/2.4.3/api/python/_modules/pyspark/ml/recommendation.html
index 2893833..fb550e6 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/ml/recommendation.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/ml/recommendation.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.ml.recommendation — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -540,12 +530,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -563,7 +553,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/ml/regression.html b/site/docs/2.4.3/api/python/_modules/pyspark/ml/regression.html
index 951cc54..283d941 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/ml/regression.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/ml/regression.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.ml.regression — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -1929,12 +1919,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -1952,7 +1942,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/ml/stat.html b/site/docs/2.4.3/api/python/_modules/pyspark/ml/stat.html
index e2012b3..2d5d394 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/ml/stat.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/ml/stat.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.ml.stat — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -381,7 +371,7 @@
<span class="sd"> - numNonzeros: a vector with the number of non-zeros for each coefficients</span>
<span class="sd"> - max: the maximum for each coefficient.</span>
<span class="sd"> - min: the minimum for each coefficient.</span>
-<span class="sd"> - normL2: the Euclidian norm for each coefficient.</span>
+<span class="sd"> - normL2: the Euclidean norm for each coefficient.</span>
<span class="sd"> - normL1: the L1 norm of each coefficient (sum of the absolute values).</span>
<span class="sd"> :param metrics:</span>
@@ -469,12 +459,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -492,7 +482,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/ml/tuning.html b/site/docs/2.4.3/api/python/_modules/pyspark/ml/tuning.html
index 5a2156d..f15a172 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/ml/tuning.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/ml/tuning.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.ml.tuning — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -840,12 +830,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -863,7 +853,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/ml/util.html b/site/docs/2.4.3/api/python/_modules/pyspark/ml/util.html
index 0f36440..4769314 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/ml/util.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/ml/util.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.ml.util — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -668,12 +658,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -691,7 +681,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/ml/wrapper.html b/site/docs/2.4.3/api/python/_modules/pyspark/ml/wrapper.html
index 1cc7c65..1e932eb 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/ml/wrapper.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/ml/wrapper.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.ml.wrapper — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -402,12 +392,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -425,7 +415,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/classification.html b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/classification.html
index 9a61173..8517cc2 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/classification.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/classification.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.mllib.classification — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -826,12 +816,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -849,7 +839,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/clustering.html b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/clustering.html
index fae70af..56d9851 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/clustering.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/clustering.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.mllib.clustering — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -1116,12 +1106,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -1139,7 +1129,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/common.html b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/common.html
index 0eb9a63..8016b05 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/common.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/common.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.mllib.common — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -218,12 +208,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -241,7 +231,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/evaluation.html b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/evaluation.html
index 6ce7df8..132358c 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/evaluation.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/evaluation.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.mllib.evaluation — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -611,12 +601,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -634,7 +624,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/feature.html b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/feature.html
index d494cd7..ff91148 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/feature.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/feature.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.mllib.feature — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -881,12 +871,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -904,7 +894,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/fpm.html b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/fpm.html
index 64b50f8..912ce64 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/fpm.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/fpm.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.mllib.fpm — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -261,12 +251,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -284,7 +274,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/linalg.html b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/linalg.html
index 90f644a..11a4f0f 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/linalg.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/linalg.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.mllib.linalg — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -326,6 +316,8 @@
<span class="sd"> DenseVector([3.0, 2.0])</span>
<span class="sd"> >>> u % 2</span>
<span class="sd"> DenseVector([1.0, 0.0])</span>
+<span class="sd"> >>> -v</span>
+<span class="sd"> DenseVector([-1.0, -2.0])</span>
<span class="sd"> """</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">ar</span><span class="p">):</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">ar</span><span class="p">,</span> <span class="nb">bytes</span><span class="p">):</span>
@@ -525,6 +517,9 @@
<span class="k">def</span> <span class="nf">__getattr__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">item</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">getattr</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">array</span><span class="p">,</span> <span class="n">item</span><span class="p">)</span>
+ <span class="k">def</span> <span class="nf">__neg__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+ <span class="k">return</span> <span class="n">DenseVector</span><span class="p">(</span><span class="o">-</span><span class="bp">self</span><span class="o">.</span><span class="n">array</span><span class="p">)</span>
+
<span class="k">def</span> <span class="nf">_delegate</span><span class="p">(</span><span class="n">op</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">func</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">other</span><span class="p">):</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">other</span><span class="p">,</span> <span class="n">DenseVector</span><span class="p">):</span>
@@ -532,7 +527,6 @@
<span class="k">return</span> <span class="n">DenseVector</span><span class="p">(</span><span class="nb">getattr</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">array</span><span class="p">,</span> <span class="n">op</span><span class="p">)(</span><span class="n">other</span><span class="p">))</span>
<span class="k">return</span> <span class="n">func</span>
- <span class="fm">__neg__</span> <span class="o">=</span> <span class="n">_delegate</span><span class="p">(</span><span class="s2">"__neg__"</span><span class="p">)</span>
<span class="fm">__add__</span> <span class="o">=</span> <span class="n">_delegate</span><span class="p">(</span><span class="s2">"__add__"</span><span class="p">)</span>
<span class="fm">__sub__</span> <span class="o">=</span> <span class="n">_delegate</span><span class="p">(</span><span class="s2">"__sub__"</span><span class="p">)</span>
<span class="fm">__mul__</span> <span class="o">=</span> <span class="n">_delegate</span><span class="p">(</span><span class="s2">"__mul__"</span><span class="p">)</span>
@@ -1437,12 +1431,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -1460,7 +1454,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/linalg/distributed.html b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/linalg/distributed.html
index fc01488..bde1ce8 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/linalg/distributed.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/linalg/distributed.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.mllib.linalg.distributed — PySpark master documentation</title>
<link rel="stylesheet" href="../../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../../" src="../../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -1445,12 +1435,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -1469,7 +1459,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/random.html b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/random.html
index 2df55d7..47d9aa0 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/random.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/random.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.mllib.random — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -484,12 +474,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -507,7 +497,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/recommendation.html b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/recommendation.html
index a471865..10fa3c0 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/recommendation.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/recommendation.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.mllib.recommendation — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -389,12 +379,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -412,7 +402,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/regression.html b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/regression.html
index fb5c27d..9e299d9 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/regression.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/regression.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.mllib.regression — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -900,12 +890,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -923,7 +913,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/stat/KernelDensity.html b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/stat/KernelDensity.html
index dcee35e..44522f7 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/stat/KernelDensity.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/stat/KernelDensity.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.mllib.stat.KernelDensity — PySpark master documentation</title>
<link rel="stylesheet" href="../../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../../" src="../../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -114,12 +104,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -137,7 +127,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/stat/distribution.html b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/stat/distribution.html
index 2d00fd9..ea8fdc8 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/stat/distribution.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/stat/distribution.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.mllib.stat.distribution — PySpark master documentation</title>
<link rel="stylesheet" href="../../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../../" src="../../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -87,12 +77,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -110,7 +100,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/stat/test.html b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/stat/test.html
index 27da70f..faf614b 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/stat/test.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/stat/test.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.mllib.stat.test — PySpark master documentation</title>
<link rel="stylesheet" href="../../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../../" src="../../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -137,12 +127,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -160,7 +150,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/tree.html b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/tree.html
index ee2bf4b..df762fe 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/tree.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/tree.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.mllib.tree — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -716,12 +706,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -739,7 +729,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/util.html b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/util.html
index ddf43c7..44d4275 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/mllib/util.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/mllib/util.html
@@ -1,32 +1,22 @@
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+ <meta charset="utf-8" />
<title>pyspark.mllib.util — PySpark master documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pyspark.css" type="text/css" />
- <script type="text/javascript">
- var DOCUMENTATION_OPTIONS = {
- URL_ROOT: '../../../',
- VERSION: 'master',
- COLLAPSE_INDEX: false,
- FILE_SUFFIX: '.html',
- HAS_SOURCE: true,
- SOURCELINK_SUFFIX: '.txt'
- };
- </script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
+ <script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
- </head>
- <body>
+ </head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
@@ -583,12 +573,12 @@
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
+ <div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
- <div><input type="text" name="q" /></div>
- <div><input type="submit" value="Go" /></div>
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
+ <input type="text" name="q" />
+ <input type="submit" value="Go" />
</form>
+ </div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@@ -606,7 +596,7 @@
</div>
<div class="footer" role="contentinfo">
© Copyright .
- Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.3.
+ Created using <a href="http://sphinx-doc.org/">Sphinx</a> 2.0.1.
</div>
</body>
</html>
\ No newline at end of file
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/profiler.html b/site/docs/2.4.3/api/python/_modules/pyspark/profiler.html
index b45d2c5..bab597c 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/profiler.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/profiler.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../_static/underscore.js"></script>
<script type="text/javascript" src="../../_static/doctools.js"></script>
<script type="text/javascript" src="../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/rdd.html b/site/docs/2.4.3/api/python/_modules/pyspark/rdd.html
index d5d7950..7283c3c 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/rdd.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/rdd.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../_static/underscore.js"></script>
<script type="text/javascript" src="../../_static/doctools.js"></script>
<script type="text/javascript" src="../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/serializers.html b/site/docs/2.4.3/api/python/_modules/pyspark/serializers.html
index fbd7584..7521581 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/serializers.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/serializers.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../_static/underscore.js"></script>
<script type="text/javascript" src="../../_static/doctools.js"></script>
<script type="text/javascript" src="../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/sql/catalog.html b/site/docs/2.4.3/api/python/_modules/pyspark/sql/catalog.html
index 9219a4d..6253737 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/sql/catalog.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/sql/catalog.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/sql/column.html b/site/docs/2.4.3/api/python/_modules/pyspark/sql/column.html
index 475b06e..cbb6e9f 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/sql/column.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/sql/column.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/sql/context.html b/site/docs/2.4.3/api/python/_modules/pyspark/sql/context.html
index 3c012f9..4c52c15 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/sql/context.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/sql/context.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/sql/dataframe.html b/site/docs/2.4.3/api/python/_modules/pyspark/sql/dataframe.html
index fb5b473..8aa7aa2 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/sql/dataframe.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/sql/dataframe.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/sql/functions.html b/site/docs/2.4.3/api/python/_modules/pyspark/sql/functions.html
index d8d3b08..ef3ac5f 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/sql/functions.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/sql/functions.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/sql/group.html b/site/docs/2.4.3/api/python/_modules/pyspark/sql/group.html
index bb42a64..664cfd9 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/sql/group.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/sql/group.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/sql/readwriter.html b/site/docs/2.4.3/api/python/_modules/pyspark/sql/readwriter.html
index 1f71873..945e78b 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/sql/readwriter.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/sql/readwriter.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/sql/session.html b/site/docs/2.4.3/api/python/_modules/pyspark/sql/session.html
index f6e9150..92dc6e1 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/sql/session.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/sql/session.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/sql/streaming.html b/site/docs/2.4.3/api/python/_modules/pyspark/sql/streaming.html
index 9b9fd5a..9c23b51 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/sql/streaming.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/sql/streaming.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/sql/types.html b/site/docs/2.4.3/api/python/_modules/pyspark/sql/types.html
index 5691f4d..5d5db38 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/sql/types.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/sql/types.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/sql/udf.html b/site/docs/2.4.3/api/python/_modules/pyspark/sql/udf.html
index a8dbaa5..585c5dd 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/sql/udf.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/sql/udf.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/sql/window.html b/site/docs/2.4.3/api/python/_modules/pyspark/sql/window.html
index 007d18c..330569b 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/sql/window.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/sql/window.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/status.html b/site/docs/2.4.3/api/python/_modules/pyspark/status.html
index 153a097..b46988c 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/status.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/status.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../_static/underscore.js"></script>
<script type="text/javascript" src="../../_static/doctools.js"></script>
<script type="text/javascript" src="../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/storagelevel.html b/site/docs/2.4.3/api/python/_modules/pyspark/storagelevel.html
index 80b3c54..0b954b4 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/storagelevel.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/storagelevel.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../_static/underscore.js"></script>
<script type="text/javascript" src="../../_static/doctools.js"></script>
<script type="text/javascript" src="../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/streaming/context.html b/site/docs/2.4.3/api/python/_modules/pyspark/streaming/context.html
index 9c79e39..b213a78 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/streaming/context.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/streaming/context.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/streaming/dstream.html b/site/docs/2.4.3/api/python/_modules/pyspark/streaming/dstream.html
index 6adc6ad..eb344ec 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/streaming/dstream.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/streaming/dstream.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/streaming/flume.html b/site/docs/2.4.3/api/python/_modules/pyspark/streaming/flume.html
index 7ff8839..37466ea 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/streaming/flume.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/streaming/flume.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/streaming/kafka.html b/site/docs/2.4.3/api/python/_modules/pyspark/streaming/kafka.html
index 9cb44d7..2a0f0c9 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/streaming/kafka.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/streaming/kafka.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/streaming/kinesis.html b/site/docs/2.4.3/api/python/_modules/pyspark/streaming/kinesis.html
index 0038fda..1018f0d 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/streaming/kinesis.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/streaming/kinesis.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/streaming/listener.html b/site/docs/2.4.3/api/python/_modules/pyspark/streaming/listener.html
index e135b4a..65d2066 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/streaming/listener.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/streaming/listener.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/_modules/pyspark/taskcontext.html b/site/docs/2.4.3/api/python/_modules/pyspark/taskcontext.html
index 692ffd5..61a922b 100644
--- a/site/docs/2.4.3/api/python/_modules/pyspark/taskcontext.html
+++ b/site/docs/2.4.3/api/python/_modules/pyspark/taskcontext.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="../../_static/underscore.js"></script>
<script type="text/javascript" src="../../_static/doctools.js"></script>
<script type="text/javascript" src="../../_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../_static/pyspark.js"></script>
<link rel="search" title="Search" href="../../search.html" />
</head><body>
diff --git a/site/docs/2.4.3/api/python/index.html b/site/docs/2.4.3/api/python/index.html
index fae417f..003ff4f 100644
--- a/site/docs/2.4.3/api/python/index.html
+++ b/site/docs/2.4.3/api/python/index.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="_static/pyspark.js"></script>
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="pyspark package" href="pyspark.html" />
@@ -59,36 +60,36 @@
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="pyspark.ml.html">pyspark.ml package</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#ml-pipeline-apis">ML Pipeline APIs</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#pyspark-ml-param-module">pyspark.ml.param module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#pyspark-ml-feature-module">pyspark.ml.feature module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#pyspark-ml-classification-module">pyspark.ml.classification module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#pyspark-ml-clustering-module">pyspark.ml.clustering module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#pyspark-ml-linalg-module">pyspark.ml.linalg module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#pyspark-ml-recommendation-module">pyspark.ml.recommendation module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#pyspark-ml-regression-module">pyspark.ml.regression module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#pyspark-ml-stat-module">pyspark.ml.stat module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#pyspark-ml-tuning-module">pyspark.ml.tuning module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#pyspark-ml-evaluation-module">pyspark.ml.evaluation module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#pyspark-ml-fpm-module">pyspark.ml.fpm module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#pyspark-ml-image-module">pyspark.ml.image module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#pyspark-ml-util-module">pyspark.ml.util module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#module-pyspark.ml">ML Pipeline APIs</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#module-pyspark.ml.param">pyspark.ml.param module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#module-pyspark.ml.feature">pyspark.ml.feature module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#module-pyspark.ml.classification">pyspark.ml.classification module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#module-pyspark.ml.clustering">pyspark.ml.clustering module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#module-pyspark.ml.linalg">pyspark.ml.linalg module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#module-pyspark.ml.recommendation">pyspark.ml.recommendation module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#module-pyspark.ml.regression">pyspark.ml.regression module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#module-pyspark.ml.stat">pyspark.ml.stat module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#module-pyspark.ml.tuning">pyspark.ml.tuning module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#module-pyspark.ml.evaluation">pyspark.ml.evaluation module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#module-pyspark.ml.fpm">pyspark.ml.fpm module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#module-pyspark.ml.image">pyspark.ml.image module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.ml.html#module-pyspark.ml.util">pyspark.ml.util module</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="pyspark.mllib.html">pyspark.mllib package</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#pyspark-mllib-classification-module">pyspark.mllib.classification module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#pyspark-mllib-clustering-module">pyspark.mllib.clustering module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#pyspark-mllib-evaluation-module">pyspark.mllib.evaluation module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#pyspark-mllib-feature-module">pyspark.mllib.feature module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#pyspark-mllib-fpm-module">pyspark.mllib.fpm module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#pyspark-mllib-linalg-module">pyspark.mllib.linalg module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#pyspark-mllib-linalg-distributed-module">pyspark.mllib.linalg.distributed module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#pyspark-mllib-random-module">pyspark.mllib.random module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#pyspark-mllib-recommendation-module">pyspark.mllib.recommendation module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#pyspark-mllib-regression-module">pyspark.mllib.regression module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#pyspark-mllib-stat-module">pyspark.mllib.stat module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#pyspark-mllib-tree-module">pyspark.mllib.tree module</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#pyspark-mllib-util-module">pyspark.mllib.util module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#module-pyspark.mllib.classification">pyspark.mllib.classification module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#module-pyspark.mllib.clustering">pyspark.mllib.clustering module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#module-pyspark.mllib.evaluation">pyspark.mllib.evaluation module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#module-pyspark.mllib.feature">pyspark.mllib.feature module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#module-pyspark.mllib.fpm">pyspark.mllib.fpm module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#module-pyspark.mllib.linalg">pyspark.mllib.linalg module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#module-pyspark.mllib.linalg.distributed">pyspark.mllib.linalg.distributed module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#module-pyspark.mllib.random">pyspark.mllib.random module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#module-pyspark.mllib.recommendation">pyspark.mllib.recommendation module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#module-pyspark.mllib.regression">pyspark.mllib.regression module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#module-pyspark.mllib.stat">pyspark.mllib.stat module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#module-pyspark.mllib.tree">pyspark.mllib.tree module</a></li>
+<li class="toctree-l2"><a class="reference internal" href="pyspark.mllib.html#module-pyspark.mllib.util">pyspark.mllib.util module</a></li>
</ul>
</li>
</ul>
diff --git a/site/docs/2.4.3/api/python/objects.inv b/site/docs/2.4.3/api/python/objects.inv
index 02f60da..457e7bc 100644
Binary files a/site/docs/2.4.3/api/python/objects.inv and b/site/docs/2.4.3/api/python/objects.inv differ
diff --git a/site/docs/2.4.3/api/python/pyspark.html b/site/docs/2.4.3/api/python/pyspark.html
index 38f273e..8315d6b 100644
--- a/site/docs/2.4.3/api/python/pyspark.html
+++ b/site/docs/2.4.3/api/python/pyspark.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="_static/pyspark.js"></script>
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="pyspark.sql module" href="pyspark.sql.html" />
diff --git a/site/docs/2.4.3/api/python/pyspark.ml.html b/site/docs/2.4.3/api/python/pyspark.ml.html
index d833df2..28ea5cc 100644
--- a/site/docs/2.4.3/api/python/pyspark.ml.html
+++ b/site/docs/2.4.3/api/python/pyspark.ml.html
@@ -13,6 +13,7 @@
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="_static/language_data.js"></script>
+ <script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="_static/pyspark.js"></script>
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="pyspark.mllib package" href="pyspark.mllib.html" />
@@ -41,47 +42,35967 @@
<div class="section" id="pyspark-ml-package">
<h1>pyspark.ml package<a class="headerlink" href="#pyspark-ml-package" title="Permalink to this headline">¶</a></h1>
-<div class="section" id="ml-pipeline-apis">
-<h2>ML Pipeline APIs<a class="headerlink" href="#ml-pipeline-apis" title="Permalink to this headline">¶</a></h2>
+<div class="section" id="module-pyspark.ml">
+<span id="ml-pipeline-apis"></span><h2>ML Pipeline APIs<a class="headerlink" href="#module-pyspark.ml" title="Permalink to this headline">¶</a></h2>
+<p>DataFrame-based machine learning APIs to let users quickly assemble and configure practical
+machine learning pipelines.</p>
+<dl class="class">
+<dt id="pyspark.ml.Transformer">
+<em class="property">class </em><code class="descclassname">pyspark.ml.</code><code class="descname">Transformer</code><a class="reference internal" href="_modules/pyspark/ml/base.html#Transformer"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.Transformer" title="Permalink to this definition">¶</a></dt>
+<dd><p>Abstract class for transformers that transform one dataset into another.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
</div>
-<div class="section" id="pyspark-ml-param-module">
-<h2>pyspark.ml.param module<a class="headerlink" href="#pyspark-ml-param-module" title="Permalink to this headline">¶</a></h2>
+<dl class="method">
+<dt id="pyspark.ml.Transformer.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Transformer.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. The default implementation creates a
+shallow copy using <code class="xref py py-func docutils literal notranslate"><span class="pre">copy.copy()</span></code>, and then copies the
+embedded and extra parameters over and returns the copy.
+Subclasses should override this method if the default approach
+is not sufficient.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Transformer.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Transformer.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Transformer.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Transformer.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Transformer.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Transformer.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Transformer.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Transformer.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Transformer.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Transformer.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Transformer.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Transformer.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Transformer.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Transformer.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Transformer.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Transformer.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Transformer.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Transformer.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.Transformer.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.Transformer.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Transformer.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Transformer.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Transformer.transform">
+<code class="descname">transform</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/base.html#Transformer.transform"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.Transformer.transform" title="Permalink to this definition">¶</a></dt>
+<dd><p>Transforms the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>transformed dataset</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.UnaryTransformer">
+<em class="property">class </em><code class="descclassname">pyspark.ml.</code><code class="descname">UnaryTransformer</code><a class="reference internal" href="_modules/pyspark/ml/base.html#UnaryTransformer"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.UnaryTransformer" title="Permalink to this definition">¶</a></dt>
+<dd><p>Abstract class for transformers that take one input column, apply transformation,
+and output the result as a new column.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.3.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.UnaryTransformer.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. The default implementation creates a
+shallow copy using <code class="xref py py-func docutils literal notranslate"><span class="pre">copy.copy()</span></code>, and then copies the
+embedded and extra parameters over and returns the copy.
+Subclasses should override this method if the default approach
+is not sufficient.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.createTransformFunc">
+<code class="descname">createTransformFunc</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/base.html#UnaryTransformer.createTransformFunc"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.UnaryTransformer.createTransformFunc" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates the transform function using the given param map. The input param map already takes
+account of the embedded param map. So the param values should be determined
+solely by the input param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.UnaryTransformer.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.UnaryTransformer.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.UnaryTransformer.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.getInputCol">
+<code class="descname">getInputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.UnaryTransformer.getInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of inputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.UnaryTransformer.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.getOutputCol">
+<code class="descname">getOutputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.UnaryTransformer.getOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of outputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.UnaryTransformer.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.UnaryTransformer.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.UnaryTransformer.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.UnaryTransformer.inputCol">
+<code class="descname">inputCol</code><em class="property"> = Param(parent='undefined', name='inputCol', doc='input column name.')</em><a class="headerlink" href="#pyspark.ml.UnaryTransformer.inputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.UnaryTransformer.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.UnaryTransformer.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.UnaryTransformer.outputCol">
+<code class="descname">outputCol</code><em class="property"> = Param(parent='undefined', name='outputCol', doc='output column name.')</em><a class="headerlink" href="#pyspark.ml.UnaryTransformer.outputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.outputDataType">
+<code class="descname">outputDataType</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/base.html#UnaryTransformer.outputDataType"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.UnaryTransformer.outputDataType" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the data type of the output column.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.UnaryTransformer.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.UnaryTransformer.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.UnaryTransformer.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.setInputCol">
+<code class="descname">setInputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.UnaryTransformer.setInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.UnaryTransformer.inputCol" title="pyspark.ml.UnaryTransformer.inputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">inputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.setOutputCol">
+<code class="descname">setOutputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.UnaryTransformer.setOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.UnaryTransformer.outputCol" title="pyspark.ml.UnaryTransformer.outputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.transform">
+<code class="descname">transform</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.UnaryTransformer.transform" title="Permalink to this definition">¶</a></dt>
+<dd><p>Transforms the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>transformed dataset</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.transformSchema">
+<code class="descname">transformSchema</code><span class="sig-paren">(</span><em>schema</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/base.html#UnaryTransformer.transformSchema"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.UnaryTransformer.transformSchema" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.UnaryTransformer.validateInputType">
+<code class="descname">validateInputType</code><span class="sig-paren">(</span><em>inputType</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/base.html#UnaryTransformer.validateInputType"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.UnaryTransformer.validateInputType" title="Permalink to this definition">¶</a></dt>
+<dd><p>Validates the input type. Throw an exception if it is invalid.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.Estimator">
+<em class="property">class </em><code class="descclassname">pyspark.ml.</code><code class="descname">Estimator</code><a class="reference internal" href="_modules/pyspark/ml/base.html#Estimator"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.Estimator" title="Permalink to this definition">¶</a></dt>
+<dd><p>Abstract class for estimators that fit models to data.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.Estimator.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Estimator.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. The default implementation creates a
+shallow copy using <code class="xref py py-func docutils literal notranslate"><span class="pre">copy.copy()</span></code>, and then copies the
+embedded and extra parameters over and returns the copy.
+Subclasses should override this method if the default approach
+is not sufficient.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Estimator.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Estimator.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Estimator.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Estimator.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Estimator.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Estimator.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Estimator.fit">
+<code class="descname">fit</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/base.html#Estimator.fit"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.Estimator.fit" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params. If a list/tuple of
+param maps is given, this calls fit on each param map and returns a list of
+models.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>fitted model(s)</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Estimator.fitMultiple">
+<code class="descname">fitMultiple</code><span class="sig-paren">(</span><em>dataset</em>, <em>paramMaps</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/base.html#Estimator.fitMultiple"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.Estimator.fitMultiple" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset for each param map in <cite>paramMaps</cite>.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a>.</p></li>
+<li><p><strong>paramMaps</strong> – A Sequence of param maps.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>A thread safe iterable which contains one model for each param map. Each
+call to <cite>next(modelIterator)</cite> will return <cite>(index, model)</cite> where model was fit
+using <cite>paramMaps[index]</cite>. <cite>index</cite> values may not be sequential.</p>
+</dd>
+</dl>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>DeveloperApi</p>
+</div>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Experimental</p>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Estimator.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Estimator.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Estimator.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Estimator.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Estimator.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Estimator.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Estimator.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Estimator.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Estimator.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Estimator.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Estimator.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Estimator.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.Estimator.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.Estimator.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Estimator.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Estimator.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.Model">
+<em class="property">class </em><code class="descclassname">pyspark.ml.</code><code class="descname">Model</code><a class="reference internal" href="_modules/pyspark/ml/base.html#Model"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.Model" title="Permalink to this definition">¶</a></dt>
+<dd><p>Abstract class for models that are fitted by estimators.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.4.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.Model.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Model.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. The default implementation creates a
+shallow copy using <code class="xref py py-func docutils literal notranslate"><span class="pre">copy.copy()</span></code>, and then copies the
+embedded and extra parameters over and returns the copy.
+Subclasses should override this method if the default approach
+is not sufficient.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Model.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Model.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Model.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Model.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Model.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Model.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Model.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Model.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Model.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Model.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Model.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Model.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Model.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Model.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Model.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Model.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Model.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Model.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.Model.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.Model.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Model.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Model.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Model.transform">
+<code class="descname">transform</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Model.transform" title="Permalink to this definition">¶</a></dt>
+<dd><p>Transforms the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>transformed dataset</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.Pipeline">
+<em class="property">class </em><code class="descclassname">pyspark.ml.</code><code class="descname">Pipeline</code><span class="sig-paren">(</span><em>stages=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/pipeline.html#Pipeline"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.Pipeline" title="Permalink to this definition">¶</a></dt>
+<dd><p>A simple pipeline, which acts as an estimator. A Pipeline consists
+of a sequence of stages, each of which is either an
+<a class="reference internal" href="#pyspark.ml.Estimator" title="pyspark.ml.Estimator"><code class="xref py py-class docutils literal notranslate"><span class="pre">Estimator</span></code></a> or a <a class="reference internal" href="#pyspark.ml.Transformer" title="pyspark.ml.Transformer"><code class="xref py py-class docutils literal notranslate"><span class="pre">Transformer</span></code></a>. When
+<a class="reference internal" href="#pyspark.ml.Pipeline.fit" title="pyspark.ml.Pipeline.fit"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Pipeline.fit()</span></code></a> is called, the stages are executed in
+order. If a stage is an <a class="reference internal" href="#pyspark.ml.Estimator" title="pyspark.ml.Estimator"><code class="xref py py-class docutils literal notranslate"><span class="pre">Estimator</span></code></a>, its
+<a class="reference internal" href="#pyspark.ml.Estimator.fit" title="pyspark.ml.Estimator.fit"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Estimator.fit()</span></code></a> method will be called on the input
+dataset to fit a model. Then the model, which is a transformer,
+will be used to transform the dataset as the input to the next
+stage. If a stage is a <a class="reference internal" href="#pyspark.ml.Transformer" title="pyspark.ml.Transformer"><code class="xref py py-class docutils literal notranslate"><span class="pre">Transformer</span></code></a>, its
+<a class="reference internal" href="#pyspark.ml.Transformer.transform" title="pyspark.ml.Transformer.transform"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Transformer.transform()</span></code></a> method will be called to produce
+the dataset for the next stage. The fitted model from a
+<a class="reference internal" href="#pyspark.ml.Pipeline" title="pyspark.ml.Pipeline"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pipeline</span></code></a> is a <a class="reference internal" href="#pyspark.ml.PipelineModel" title="pyspark.ml.PipelineModel"><code class="xref py py-class docutils literal notranslate"><span class="pre">PipelineModel</span></code></a>, which
+consists of fitted models and transformers, corresponding to the
+pipeline stages. If stages is an empty list, the pipeline acts as an
+identity transformer.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.Pipeline.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/pipeline.html#Pipeline.copy"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.Pipeline.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra parameters</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>new instance</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.4.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Pipeline.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Pipeline.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Pipeline.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Pipeline.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Pipeline.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Pipeline.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Pipeline.fit">
+<code class="descname">fit</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Pipeline.fit" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params. If a list/tuple of
+param maps is given, this calls fit on each param map and returns a list of
+models.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>fitted model(s)</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Pipeline.fitMultiple">
+<code class="descname">fitMultiple</code><span class="sig-paren">(</span><em>dataset</em>, <em>paramMaps</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Pipeline.fitMultiple" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset for each param map in <cite>paramMaps</cite>.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a>.</p></li>
+<li><p><strong>paramMaps</strong> – A Sequence of param maps.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>A thread safe iterable which contains one model for each param map. Each
+call to <cite>next(modelIterator)</cite> will return <cite>(index, model)</cite> where model was fit
+using <cite>paramMaps[index]</cite>. <cite>index</cite> values may not be sequential.</p>
+</dd>
+</dl>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>DeveloperApi</p>
+</div>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Experimental</p>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Pipeline.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Pipeline.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Pipeline.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Pipeline.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Pipeline.getStages">
+<code class="descname">getStages</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/pipeline.html#Pipeline.getStages"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.Pipeline.getStages" title="Permalink to this definition">¶</a></dt>
+<dd><p>Get pipeline stages.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Pipeline.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Pipeline.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Pipeline.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Pipeline.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Pipeline.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Pipeline.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Pipeline.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Pipeline.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.Pipeline.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Pipeline.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.Pipeline.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.Pipeline.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.Pipeline.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/pipeline.html#Pipeline.read"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.Pipeline.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Pipeline.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Pipeline.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Pipeline.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.Pipeline.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Pipeline.setParams">
+<code class="descname">setParams</code><span class="sig-paren">(</span><em>self</em>, <em>stages=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/pipeline.html#Pipeline.setParams"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.Pipeline.setParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets params for Pipeline.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Pipeline.setStages">
+<code class="descname">setStages</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/pipeline.html#Pipeline.setStages"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.Pipeline.setStages" title="Permalink to this definition">¶</a></dt>
+<dd><p>Set pipeline stages.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>value</strong> – a list of transformers or estimators</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>the pipeline instance</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.Pipeline.stages">
+<code class="descname">stages</code><em class="property"> = Param(parent='undefined', name='stages', doc='a list of pipeline stages')</em><a class="headerlink" href="#pyspark.ml.Pipeline.stages" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.Pipeline.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/pipeline.html#Pipeline.write"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.Pipeline.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.PipelineModel">
+<em class="property">class </em><code class="descclassname">pyspark.ml.</code><code class="descname">PipelineModel</code><span class="sig-paren">(</span><em>stages</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/pipeline.html#PipelineModel"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.PipelineModel" title="Permalink to this definition">¶</a></dt>
+<dd><p>Represents a compiled pipeline with transformers and fitted models.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.PipelineModel.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/pipeline.html#PipelineModel.copy"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.PipelineModel.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra parameters</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>new instance</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.4.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.PipelineModel.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.PipelineModel.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.PipelineModel.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.PipelineModel.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.PipelineModel.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.PipelineModel.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.PipelineModel.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.PipelineModel.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.PipelineModel.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.PipelineModel.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.PipelineModel.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.PipelineModel.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.PipelineModel.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.PipelineModel.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.PipelineModel.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.PipelineModel.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.PipelineModel.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.PipelineModel.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.PipelineModel.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.PipelineModel.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.PipelineModel.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.PipelineModel.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.PipelineModel.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/pipeline.html#PipelineModel.read"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.PipelineModel.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.PipelineModel.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.PipelineModel.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.PipelineModel.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.PipelineModel.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.PipelineModel.transform">
+<code class="descname">transform</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.PipelineModel.transform" title="Permalink to this definition">¶</a></dt>
+<dd><p>Transforms the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>transformed dataset</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.PipelineModel.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/pipeline.html#PipelineModel.write"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.PipelineModel.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+</dd></dl>
+
+</dd></dl>
+
+</div>
+<div class="section" id="module-pyspark.ml.param">
+<span id="pyspark-ml-param-module"></span><h2>pyspark.ml.param module<a class="headerlink" href="#module-pyspark.ml.param" title="Permalink to this headline">¶</a></h2>
+<dl class="class">
+<dt id="pyspark.ml.param.Param">
+<em class="property">class </em><code class="descclassname">pyspark.ml.param.</code><code class="descname">Param</code><span class="sig-paren">(</span><em>parent</em>, <em>name</em>, <em>doc</em>, <em>typeConverter=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#Param"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.Param" title="Permalink to this definition">¶</a></dt>
+<dd><p>A param with self-contained documentation.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.param.Params">
+<em class="property">class </em><code class="descclassname">pyspark.ml.param.</code><code class="descname">Params</code><a class="reference internal" href="_modules/pyspark/ml/param.html#Params"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.Params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Components that take parameters. This also provides an internal
+param map to store parameter values attached to the instance.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.param.Params.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#Params.copy"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.Params.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. The default implementation creates a
+shallow copy using <code class="xref py py-func docutils literal notranslate"><span class="pre">copy.copy()</span></code>, and then copies the
+embedded and extra parameters over and returns the copy.
+Subclasses should override this method if the default approach
+is not sufficient.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.param.Params.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#Params.explainParam"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.Params.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.param.Params.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#Params.explainParams"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.Params.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.param.Params.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#Params.extractParamMap"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.Params.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.param.Params.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#Params.getOrDefault"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.Params.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.param.Params.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#Params.getParam"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.Params.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.param.Params.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#Params.hasDefault"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.Params.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.param.Params.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#Params.hasParam"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.Params.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.param.Params.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#Params.isDefined"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.Params.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.param.Params.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#Params.isSet"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.Params.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.param.Params.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.param.Params.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<a class="reference internal" href="#pyspark.ml.param.Param" title="pyspark.ml.param.Param"><code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.param.Params.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#Params.set"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.Params.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.param.TypeConverters">
+<em class="property">class </em><code class="descclassname">pyspark.ml.param.</code><code class="descname">TypeConverters</code><a class="reference internal" href="_modules/pyspark/ml/param.html#TypeConverters"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.TypeConverters" title="Permalink to this definition">¶</a></dt>
+<dd><div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>DeveloperApi</p>
+</div>
+<p>Factory methods for common type conversion functions for <cite>Param.typeConverter</cite>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+<dl class="staticmethod">
+<dt id="pyspark.ml.param.TypeConverters.identity">
+<em class="property">static </em><code class="descname">identity</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#TypeConverters.identity"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.TypeConverters.identity" title="Permalink to this definition">¶</a></dt>
+<dd><p>Dummy converter that just returns value.</p>
+</dd></dl>
+
+<dl class="staticmethod">
+<dt id="pyspark.ml.param.TypeConverters.toBoolean">
+<em class="property">static </em><code class="descname">toBoolean</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#TypeConverters.toBoolean"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.TypeConverters.toBoolean" title="Permalink to this definition">¶</a></dt>
+<dd><p>Convert a value to a boolean, if possible.</p>
+</dd></dl>
+
+<dl class="staticmethod">
+<dt id="pyspark.ml.param.TypeConverters.toFloat">
+<em class="property">static </em><code class="descname">toFloat</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#TypeConverters.toFloat"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.TypeConverters.toFloat" title="Permalink to this definition">¶</a></dt>
+<dd><p>Convert a value to a float, if possible.</p>
+</dd></dl>
+
+<dl class="staticmethod">
+<dt id="pyspark.ml.param.TypeConverters.toInt">
+<em class="property">static </em><code class="descname">toInt</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#TypeConverters.toInt"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.TypeConverters.toInt" title="Permalink to this definition">¶</a></dt>
+<dd><p>Convert a value to an int, if possible.</p>
+</dd></dl>
+
+<dl class="staticmethod">
+<dt id="pyspark.ml.param.TypeConverters.toList">
+<em class="property">static </em><code class="descname">toList</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#TypeConverters.toList"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.TypeConverters.toList" title="Permalink to this definition">¶</a></dt>
+<dd><p>Convert a value to a list, if possible.</p>
+</dd></dl>
+
+<dl class="staticmethod">
+<dt id="pyspark.ml.param.TypeConverters.toListFloat">
+<em class="property">static </em><code class="descname">toListFloat</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#TypeConverters.toListFloat"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.TypeConverters.toListFloat" title="Permalink to this definition">¶</a></dt>
+<dd><p>Convert a value to list of floats, if possible.</p>
+</dd></dl>
+
+<dl class="staticmethod">
+<dt id="pyspark.ml.param.TypeConverters.toListInt">
+<em class="property">static </em><code class="descname">toListInt</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#TypeConverters.toListInt"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.TypeConverters.toListInt" title="Permalink to this definition">¶</a></dt>
+<dd><p>Convert a value to list of ints, if possible.</p>
+</dd></dl>
+
+<dl class="staticmethod">
+<dt id="pyspark.ml.param.TypeConverters.toListString">
+<em class="property">static </em><code class="descname">toListString</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#TypeConverters.toListString"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.TypeConverters.toListString" title="Permalink to this definition">¶</a></dt>
+<dd><p>Convert a value to list of strings, if possible.</p>
+</dd></dl>
+
+<dl class="staticmethod">
+<dt id="pyspark.ml.param.TypeConverters.toMatrix">
+<em class="property">static </em><code class="descname">toMatrix</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#TypeConverters.toMatrix"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.TypeConverters.toMatrix" title="Permalink to this definition">¶</a></dt>
+<dd><p>Convert a value to a MLlib Matrix, if possible.</p>
+</dd></dl>
+
+<dl class="staticmethod">
+<dt id="pyspark.ml.param.TypeConverters.toString">
+<em class="property">static </em><code class="descname">toString</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#TypeConverters.toString"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.TypeConverters.toString" title="Permalink to this definition">¶</a></dt>
+<dd><p>Convert a value to a string, if possible.</p>
+</dd></dl>
+
+<dl class="staticmethod">
+<dt id="pyspark.ml.param.TypeConverters.toVector">
+<em class="property">static </em><code class="descname">toVector</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/param.html#TypeConverters.toVector"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.param.TypeConverters.toVector" title="Permalink to this definition">¶</a></dt>
+<dd><p>Convert a value to a MLlib Vector, if possible.</p>
+</dd></dl>
+
+</dd></dl>
+
+</div>
+<div class="section" id="module-pyspark.ml.feature">
+<span id="pyspark-ml-feature-module"></span><h2>pyspark.ml.feature module<a class="headerlink" href="#module-pyspark.ml.feature" title="Permalink to this headline">¶</a></h2>
+<dl class="class">
+<dt id="pyspark.ml.feature.Binarizer">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">Binarizer</code><span class="sig-paren">(</span><em>threshold=0.0</em>, <em>inputCol=None</em>, <em>outputCol=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#Binarizer"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.Binarizer" title="Permalink to this definition">¶</a></dt>
+<dd><p>Binarize a column of continuous features given a threshold.</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">df</span> <span class="o">=</span> <span class="n">spark</span><span class="o">.</span><span class="n">createDataFrame</span><span class="p">([(</span><span class="mf">0.5</span><span class="p">,)],</span> <span class="p">[</span><span class="s2">"values"</span><span class="p">])</span>
+<span class="gp">>>> </span><span class="n">binarizer</span> <span class="o">=</span> <span class="n">Binarizer</span><span class="p">(</span><span class="n">threshold</span><span class="o">=</span><span class="mf">1.0</span><span class="p">,</span> <span class="n">inputCol</span><span class="o">=</span><span class="s2">"values"</span><span class="p">,</span> <span class="n">outputCol</span><span class="o">=</span><span class="s2">"features"</span><span class [...]
+<span class="gp">>>> </span><span class="n">binarizer</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">features</span>
+<span class="go">0.0</span>
+<span class="gp">>>> </span><span class="n">binarizer</span><span class="o">.</span><span class="n">setParams</span><span class="p">(</span><span class="n">outputCol</span><span class="o">=</span><span class="s2">"freqs"</span><span class="p">)</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">. [...]
+<span class="go">0.0</span>
+<span class="gp">>>> </span><span class="n">params</span> <span class="o">=</span> <span class="p">{</span><span class="n">binarizer</span><span class="o">.</span><span class="n">threshold</span><span class="p">:</span> <span class="o">-</span><span class="mf">0.5</span><span class="p">,</span> <span class="n">binarizer</span><span class="o">.</span><span class="n">outputCol</span><span class="p">:</span> <span class="s2">"vector"</span><span class="p">}</span>
+<span class="gp">>>> </span><span class="n">binarizer</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">,</span> <span class="n">params</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">vector</span>
+<span class="go">1.0</span>
+<span class="gp">>>> </span><span class="n">binarizerPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/binarizer"</span>
+<span class="gp">>>> </span><span class="n">binarizer</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">binarizerPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedBinarizer</span> <span class="o">=</span> <span class="n">Binarizer</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">binarizerPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedBinarizer</span><span class="o">.</span><span class="n">getThreshold</span><span class="p">()</span> <span class="o">==</span> <span class="n">binarizer</span><span class="o">.</span><span class="n">getThreshold</span><span class="p">()</span>
+<span class="go">True</span>
+</pre></div>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.4.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.getInputCol">
+<code class="descname">getInputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.getInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of inputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.getOutputCol">
+<code class="descname">getOutputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.getOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of outputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.getThreshold">
+<code class="descname">getThreshold</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#Binarizer.getThreshold"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.Binarizer.getThreshold" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of threshold or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.4.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.Binarizer.inputCol">
+<code class="descname">inputCol</code><em class="property"> = Param(parent='undefined', name='inputCol', doc='input column name.')</em><a class="headerlink" href="#pyspark.ml.feature.Binarizer.inputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.Binarizer.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.Binarizer.outputCol">
+<code class="descname">outputCol</code><em class="property"> = Param(parent='undefined', name='outputCol', doc='output column name.')</em><a class="headerlink" href="#pyspark.ml.feature.Binarizer.outputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.Binarizer.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.Binarizer.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.Binarizer.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.setInputCol">
+<code class="descname">setInputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.setInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.Binarizer.inputCol" title="pyspark.ml.feature.Binarizer.inputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">inputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.setOutputCol">
+<code class="descname">setOutputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.setOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.Binarizer.outputCol" title="pyspark.ml.feature.Binarizer.outputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.setParams">
+<code class="descname">setParams</code><span class="sig-paren">(</span><em>self</em>, <em>threshold=0.0</em>, <em>inputCol=None</em>, <em>outputCol=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#Binarizer.setParams"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.Binarizer.setParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets params for this Binarizer.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.4.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.setThreshold">
+<code class="descname">setThreshold</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#Binarizer.setThreshold"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.Binarizer.setThreshold" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.Binarizer.threshold" title="pyspark.ml.feature.Binarizer.threshold"><code class="xref py py-attr docutils literal notranslate"><span class="pre">threshold</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.4.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.Binarizer.threshold">
+<code class="descname">threshold</code><em class="property"> = Param(parent='undefined', name='threshold', doc='threshold in binary classification prediction, in range [0, 1]')</em><a class="headerlink" href="#pyspark.ml.feature.Binarizer.threshold" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.transform">
+<code class="descname">transform</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.transform" title="Permalink to this definition">¶</a></dt>
+<dd><p>Transforms the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>transformed dataset</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Binarizer.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Binarizer.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">BucketedRandomProjectionLSH</code><span class="sig-paren">(</span><em>inputCol=None</em>, <em>outputCol=None</em>, <em>seed=None</em>, <em>numHashTables=1</em>, <em>bucketLength=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#BucketedRandomProjectionLSH"><span class="viewcode-link">[source]</span></a><a class="headerlin [...]
+<dd><div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Experimental</p>
+</div>
+<p>LSH class for Euclidean distance metrics.
+The input is dense or sparse vectors, each of which represents a point in the Euclidean
+distance space. The output will be vectors of configurable dimension. Hash values in the same
+dimension are calculated by the same hash function.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
+<p><a class="reference external" href="https://en.wikipedia.org/wiki/Locality-sensitive_hashing#Stable_distributions">Stable Distributions</a></p>
+</div>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
+<p><a class="reference external" href="https://arxiv.org/abs/1408.2927">Hashing for Similarity Search: A Survey</a></p>
+</div>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">pyspark.ml.linalg</span> <span class="k">import</span> <span class="n">Vectors</span>
+<span class="gp">>>> </span><span class="kn">from</span> <span class="nn">pyspark.sql.functions</span> <span class="k">import</span> <span class="n">col</span>
+<span class="gp">>>> </span><span class="n">data</span> <span class="o">=</span> <span class="p">[(</span><span class="mi">0</span><span class="p">,</span> <span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([</span><span class="o">-</span><span class="mf">1.0</span><span class="p">,</span> <span class="o">-</span><span class="mf">1.0</span> <span class="p">]),),</span>
+<span class="gp">... </span> <span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([</span><span class="o">-</span><span class="mf">1.0</span><span class="p">,</span> <span class="mf">1.0</span> <span class="p">]),),</span>
+<span class="gp">... </span> <span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([</span><span class="mf">1.0</span><span class="p">,</span> <span class="o">-</span><span class="mf">1.0</span> <span class="p">]),),</span>
+<span class="gp">... </span> <span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([</span><span class="mf">1.0</span><span class="p">,</span> <span class="mf">1.0</span><span class="p">]),)]</span>
+<span class="gp">>>> </span><span class="n">df</span> <span class="o">=</span> <span class="n">spark</span><span class="o">.</span><span class="n">createDataFrame</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="p">[</span><span class="s2">"id"</span><span class="p">,</span> <span class="s2">"features"</span><span class="p">])</span>
+<span class="gp">>>> </span><span class="n">brp</span> <span class="o">=</span> <span class="n">BucketedRandomProjectionLSH</span><span class="p">(</span><span class="n">inputCol</span><span class="o">=</span><span class="s2">"features"</span><span class="p">,</span> <span class="n">outputCol</span><span class="o">=</span><span class="s2">"hashes"</span><span class="p">,</span>
+<span class="gp">... </span> <span class="n">seed</span><span class="o">=</span><span class="mi">12345</span><span class="p">,</span> <span class="n">bucketLength</span><span class="o">=</span><span class="mf">1.0</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">model</span> <span class="o">=</span> <span class="n">brp</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">df</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span>
+<span class="go">Row(id=0, features=DenseVector([-1.0, -1.0]), hashes=[DenseVector([-1.0])])</span>
+<span class="gp">>>> </span><span class="n">data2</span> <span class="o">=</span> <span class="p">[(</span><span class="mi">4</span><span class="p">,</span> <span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([</span><span class="mf">2.0</span><span class="p">,</span> <span class="mf">2.0</span> <span class="p">]),),</span>
+<span class="gp">... </span> <span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([</span><span class="mf">2.0</span><span class="p">,</span> <span class="mf">3.0</span> <span class="p">]),),</span>
+<span class="gp">... </span> <span class="p">(</span><span class="mi">6</span><span class="p">,</span> <span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([</span><span class="mf">3.0</span><span class="p">,</span> <span class="mf">2.0</span> <span class="p">]),),</span>
+<span class="gp">... </span> <span class="p">(</span><span class="mi">7</span><span class="p">,</span> <span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([</span><span class="mf">3.0</span><span class="p">,</span> <span class="mf">3.0</span><span class="p">]),)]</span>
+<span class="gp">>>> </span><span class="n">df2</span> <span class="o">=</span> <span class="n">spark</span><span class="o">.</span><span class="n">createDataFrame</span><span class="p">(</span><span class="n">data2</span><span class="p">,</span> <span class="p">[</span><span class="s2">"id"</span><span class="p">,</span> <span class="s2">"features"</span><span class="p">])</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">approxNearestNeighbors</span><span class="p">(</span><span class="n">df2</span><span class="p">,</span> <span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([</span><span class="mf">1.0</span><span class="p">,</span> <span class="mf">2.0</span><span class="p">]),</span> <span class="mi">1</span><span class="p">)</span><span class="o">.</sp [...]
+<span class="go">[Row(id=4, features=DenseVector([2.0, 2.0]), hashes=[DenseVector([1.0])], distCol=1.0)]</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">approxSimilarityJoin</span><span class="p">(</span><span class="n">df</span><span class="p">,</span> <span class="n">df2</span><span class="p">,</span> <span class="mf">3.0</span><span class="p">,</span> <span class="n">distCol</span><span class="o">=</span><span class="s2">"EuclideanDistance"</span><span class="p">)</span><span class="o">.</span><span class="n">select</span [...]
+<span class="gp">... </span> <span class="n">col</span><span class="p">(</span><span class="s2">"datasetA.id"</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">"idA"</span><span class="p">),</span>
+<span class="gp">... </span> <span class="n">col</span><span class="p">(</span><span class="s2">"datasetB.id"</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">"idB"</span><span class="p">),</span>
+<span class="gp">... </span> <span class="n">col</span><span class="p">(</span><span class="s2">"EuclideanDistance"</span><span class="p">))</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
+<span class="go">+---+---+-----------------+</span>
+<span class="go">|idA|idB|EuclideanDistance|</span>
+<span class="go">+---+---+-----------------+</span>
+<span class="go">| 3| 6| 2.23606797749979|</span>
+<span class="go">+---+---+-----------------+</span>
+<span class="gp">...</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">approxSimilarityJoin</span><span class="p">(</span><span class="n">df</span><span class="p">,</span> <span class="n">df2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="n">distCol</span><span class="o">=</span><span class="s2">"EuclideanDistance"</span><span class="p">)</span><span class="o">.</span><span class="n">select</span>< [...]
+<span class="gp">... </span> <span class="n">col</span><span class="p">(</span><span class="s2">"datasetA.id"</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">"idA"</span><span class="p">),</span>
+<span class="gp">... </span> <span class="n">col</span><span class="p">(</span><span class="s2">"datasetB.id"</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">"idB"</span><span class="p">),</span>
+<span class="gp">... </span> <span class="n">col</span><span class="p">(</span><span class="s2">"EuclideanDistance"</span><span class="p">))</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
+<span class="go">+---+---+-----------------+</span>
+<span class="go">|idA|idB|EuclideanDistance|</span>
+<span class="go">+---+---+-----------------+</span>
+<span class="go">| 3| 6| 2.23606797749979|</span>
+<span class="go">+---+---+-----------------+</span>
+<span class="gp">...</span>
+<span class="gp">>>> </span><span class="n">brpPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/brp"</span>
+<span class="gp">>>> </span><span class="n">brp</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">brpPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">brp2</span> <span class="o">=</span> <span class="n">BucketedRandomProjectionLSH</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">brpPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">brp2</span><span class="o">.</span><span class="n">getBucketLength</span><span class="p">()</span> <span class="o">==</span> <span class="n">brp</span><span class="o">.</span><span class="n">getBucketLength</span><span class="p">()</span>
+<span class="go">True</span>
+<span class="gp">>>> </span><span class="n">modelPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/brp-model"</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">modelPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">model2</span> <span class="o">=</span> <span class="n">BucketedRandomProjectionLSHModel</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">modelPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">hashes</span> <span class="o">==</span> <span class="n">model2</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span cl [...]
+<span class="go">True</span>
+</pre></div>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+<dl class="attribute">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.bucketLength">
+<code class="descname">bucketLength</code><em class="property"> = Param(parent='undefined', name='bucketLength', doc='the length of each hash bucket, a larger bucket lowers the false negative rate.')</em><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.bucketLength" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.fit">
+<code class="descname">fit</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.fit" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params. If a list/tuple of
+param maps is given, this calls fit on each param map and returns a list of
+models.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>fitted model(s)</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.fitMultiple">
+<code class="descname">fitMultiple</code><span class="sig-paren">(</span><em>dataset</em>, <em>paramMaps</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.fitMultiple" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset for each param map in <cite>paramMaps</cite>.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a>.</p></li>
+<li><p><strong>paramMaps</strong> – A Sequence of param maps.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>A thread safe iterable which contains one model for each param map. Each
+call to <cite>next(modelIterator)</cite> will return <cite>(index, model)</cite> where model was fit
+using <cite>paramMaps[index]</cite>. <cite>index</cite> values may not be sequential.</p>
+</dd>
+</dl>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>DeveloperApi</p>
+</div>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Experimental</p>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.getBucketLength">
+<code class="descname">getBucketLength</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#BucketedRandomProjectionLSH.getBucketLength"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.getBucketLength" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of bucketLength or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.getInputCol">
+<code class="descname">getInputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.getInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of inputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.getNumHashTables">
+<code class="descname">getNumHashTables</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.getNumHashTables" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of numHashTables or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.getOutputCol">
+<code class="descname">getOutputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.getOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of outputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.getSeed">
+<code class="descname">getSeed</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.getSeed" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of seed or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.inputCol">
+<code class="descname">inputCol</code><em class="property"> = Param(parent='undefined', name='inputCol', doc='input column name.')</em><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.inputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.numHashTables">
+<code class="descname">numHashTables</code><em class="property"> = Param(parent='undefined', name='numHashTables', doc='number of hash tables, where increasing number of hash tables lowers the false negative rate, and decreasing it improves the running performance.')</em><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.numHashTables" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.outputCol">
+<code class="descname">outputCol</code><em class="property"> = Param(parent='undefined', name='outputCol', doc='output column name.')</em><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.outputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.seed">
+<code class="descname">seed</code><em class="property"> = Param(parent='undefined', name='seed', doc='random seed.')</em><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.seed" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.setBucketLength">
+<code class="descname">setBucketLength</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#BucketedRandomProjectionLSH.setBucketLength"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.setBucketLength" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.bucketLength" title="pyspark.ml.feature.BucketedRandomProjectionLSH.bucketLength"><code class="xref py py-attr docutils literal notranslate"><span class="pre">bucketLength</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.setInputCol">
+<code class="descname">setInputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.setInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.inputCol" title="pyspark.ml.feature.BucketedRandomProjectionLSH.inputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">inputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.setNumHashTables">
+<code class="descname">setNumHashTables</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.setNumHashTables" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.numHashTables" title="pyspark.ml.feature.BucketedRandomProjectionLSH.numHashTables"><code class="xref py py-attr docutils literal notranslate"><span class="pre">numHashTables</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.setOutputCol">
+<code class="descname">setOutputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.setOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.outputCol" title="pyspark.ml.feature.BucketedRandomProjectionLSH.outputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.setParams">
+<code class="descname">setParams</code><span class="sig-paren">(</span><em>self</em>, <em>inputCol=None</em>, <em>outputCol=None</em>, <em>seed=None</em>, <em>numHashTables=1</em>, <em>bucketLength=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#BucketedRandomProjectionLSH.setParams"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.setParams" title="Perm [...]
+<dd><p>Sets params for this BucketedRandomProjectionLSH.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.setSeed">
+<code class="descname">setSeed</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.setSeed" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.seed" title="pyspark.ml.feature.BucketedRandomProjectionLSH.seed"><code class="xref py py-attr docutils literal notranslate"><span class="pre">seed</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSH.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSH.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">BucketedRandomProjectionLSHModel</code><span class="sig-paren">(</span><em>java_model=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#BucketedRandomProjectionLSHModel"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel" title="Permalink to this [...]
+<dd><div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Experimental</p>
+</div>
+<p>Model fitted by <a class="reference internal" href="#pyspark.ml.feature.BucketedRandomProjectionLSH" title="pyspark.ml.feature.BucketedRandomProjectionLSH"><code class="xref py py-class docutils literal notranslate"><span class="pre">BucketedRandomProjectionLSH</span></code></a>, where multiple random vectors are
+stored. The vectors are normalized to be unit vectors and each vector is used in a hash
+function: <span class="math notranslate nohighlight">\(h_i(x) = floor(r_i \cdot x / bucketLength)\)</span> where <span class="math notranslate nohighlight">\(r_i\)</span> is the
+i-th random unit vector. The number of buckets will be <cite>(max L2 norm of input vectors) /
+bucketLength</cite>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel.approxNearestNeighbors">
+<code class="descname">approxNearestNeighbors</code><span class="sig-paren">(</span><em>dataset</em>, <em>key</em>, <em>numNearestNeighbors</em>, <em>distCol='distCol'</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel.approxNearestNeighbors" title="Permalink to this definition">¶</a></dt>
+<dd><p>Given a large dataset and an item, approximately find at most k items which have the
+closest distance to the item. If the <code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code> is missing, the method will
+transform the data; if the <code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code> exists, it will use that. This allows
+caching of the transformed data when necessary.</p>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>This method is experimental and will likely change behavior in the next release.</p>
+</div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – The dataset to search for nearest neighbors of the key.</p></li>
+<li><p><strong>key</strong> – Feature vector representing the item to search for.</p></li>
+<li><p><strong>numNearestNeighbors</strong> – The maximum number of nearest neighbors.</p></li>
+<li><p><strong>distCol</strong> – Output column for storing the distance between each result row and the key.
+Use “distCol” as default value if it’s not specified.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>A dataset containing at most k items closest to the key. A column “distCol” is
+added to show the distance between each row and the key.</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel.approxSimilarityJoin">
+<code class="descname">approxSimilarityJoin</code><span class="sig-paren">(</span><em>datasetA</em>, <em>datasetB</em>, <em>threshold</em>, <em>distCol='distCol'</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel.approxSimilarityJoin" title="Permalink to this definition">¶</a></dt>
+<dd><p>Join two datasets to approximately find all pairs of rows whose distance are smaller than
+the threshold. If the <code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code> is missing, the method will transform the data;
+if the <code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code> exists, it will use that. This allows caching of the
+transformed data when necessary.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>datasetA</strong> – One of the datasets to join.</p></li>
+<li><p><strong>datasetB</strong> – Another dataset to join.</p></li>
+<li><p><strong>threshold</strong> – The threshold for the distance of row pairs.</p></li>
+<li><p><strong>distCol</strong> – Output column for storing the distance between each pair of rows. Use
+“distCol” as default value if it’s not specified.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>A joined dataset containing pairs of rows. The original rows are in columns
+“datasetA” and “datasetB”, and a column “distCol” is added to show the distance
+between each pair.</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel.transform">
+<code class="descname">transform</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel.transform" title="Permalink to this definition">¶</a></dt>
+<dd><p>Transforms the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>transformed dataset</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.BucketedRandomProjectionLSHModel.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.BucketedRandomProjectionLSHModel.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.Bucketizer">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">Bucketizer</code><span class="sig-paren">(</span><em>splits=None</em>, <em>inputCol=None</em>, <em>outputCol=None</em>, <em>handleInvalid='error'</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#Bucketizer"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.Bucketizer" title="Permal [...]
+<dd><p>Maps a column of continuous features to a column of feature buckets.</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">values</span> <span class="o">=</span> <span class="p">[(</span><span class="mf">0.1</span><span class="p">,),</span> <span class="p">(</span><span class="mf">0.4</span><span class="p">,),</span> <span class="p">(</span><span class="mf">1.2</span><span class="p">,),</span> <span class="p">(</span><span class="mf">1.5</span><span class="p">,),</span> <sp [...]
+<span class="gp">>>> </span><span class="n">df</span> <span class="o">=</span> <span class="n">spark</span><span class="o">.</span><span class="n">createDataFrame</span><span class="p">(</span><span class="n">values</span><span class="p">,</span> <span class="p">[</span><span class="s2">"values"</span><span class="p">])</span>
+<span class="gp">>>> </span><span class="n">bucketizer</span> <span class="o">=</span> <span class="n">Bucketizer</span><span class="p">(</span><span class="n">splits</span><span class="o">=</span><span class="p">[</span><span class="o">-</span><span class="nb">float</span><span class="p">(</span><span class="s2">"inf"</span><span class="p">),</span> <span class="mf">0.5</span><span class="p">,</span> <span class="mf">1.4</span><span class="p">,</span> <span class="nb" [...]
+<span class="gp">... </span> <span class="n">inputCol</span><span class="o">=</span><span class="s2">"values"</span><span class="p">,</span> <span class="n">outputCol</span><span class="o">=</span><span class="s2">"buckets"</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">bucketed</span> <span class="o">=</span> <span class="n">bucketizer</span><span class="o">.</span><span class="n">setHandleInvalid</span><span class="p">(</span><span class="s2">"keep"</span><span class="p">)</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">collect</span><span class="p">()</span>
+<span class="gp">>>> </span><span class="nb">len</span><span class="p">(</span><span class="n">bucketed</span><span class="p">)</span>
+<span class="go">6</span>
+<span class="gp">>>> </span><span class="n">bucketed</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">buckets</span>
+<span class="go">0.0</span>
+<span class="gp">>>> </span><span class="n">bucketed</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">buckets</span>
+<span class="go">0.0</span>
+<span class="gp">>>> </span><span class="n">bucketed</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span><span class="o">.</span><span class="n">buckets</span>
+<span class="go">1.0</span>
+<span class="gp">>>> </span><span class="n">bucketed</span><span class="p">[</span><span class="mi">3</span><span class="p">]</span><span class="o">.</span><span class="n">buckets</span>
+<span class="go">2.0</span>
+<span class="gp">>>> </span><span class="n">bucketizer</span><span class="o">.</span><span class="n">setParams</span><span class="p">(</span><span class="n">outputCol</span><span class="o">=</span><span class="s2">"b"</span><span class="p">)</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</s [...]
+<span class="go">0.0</span>
+<span class="gp">>>> </span><span class="n">bucketizerPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/bucketizer"</span>
+<span class="gp">>>> </span><span class="n">bucketizer</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">bucketizerPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedBucketizer</span> <span class="o">=</span> <span class="n">Bucketizer</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">bucketizerPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedBucketizer</span><span class="o">.</span><span class="n">getSplits</span><span class="p">()</span> <span class="o">==</span> <span class="n">bucketizer</span><span class="o">.</span><span class="n">getSplits</span><span class="p">()</span>
+<span class="go">True</span>
+<span class="gp">>>> </span><span class="n">bucketed</span> <span class="o">=</span> <span class="n">bucketizer</span><span class="o">.</span><span class="n">setHandleInvalid</span><span class="p">(</span><span class="s2">"skip"</span><span class="p">)</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">collect</span><span class="p">()</span>
+<span class="gp">>>> </span><span class="nb">len</span><span class="p">(</span><span class="n">bucketed</span><span class="p">)</span>
+<span class="go">4</span>
+</pre></div>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.4.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.getHandleInvalid">
+<code class="descname">getHandleInvalid</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.getHandleInvalid" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of handleInvalid or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.getInputCol">
+<code class="descname">getInputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.getInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of inputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.getOutputCol">
+<code class="descname">getOutputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.getOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of outputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.getSplits">
+<code class="descname">getSplits</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#Bucketizer.getSplits"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.getSplits" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of threshold or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.4.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.Bucketizer.handleInvalid">
+<code class="descname">handleInvalid</code><em class="property"> = Param(parent='undefined', name='handleInvalid', doc="how to handle invalid entries. Options are 'skip' (filter out rows with invalid values), 'error' (throw an error), or 'keep' (keep invalid values in a special additional bucket).")</em><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.handleInvalid" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.Bucketizer.inputCol">
+<code class="descname">inputCol</code><em class="property"> = Param(parent='undefined', name='inputCol', doc='input column name.')</em><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.inputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.Bucketizer.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.Bucketizer.outputCol">
+<code class="descname">outputCol</code><em class="property"> = Param(parent='undefined', name='outputCol', doc='output column name.')</em><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.outputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.Bucketizer.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.Bucketizer.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.setHandleInvalid">
+<code class="descname">setHandleInvalid</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.setHandleInvalid" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.Bucketizer.handleInvalid" title="pyspark.ml.feature.Bucketizer.handleInvalid"><code class="xref py py-attr docutils literal notranslate"><span class="pre">handleInvalid</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.setInputCol">
+<code class="descname">setInputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.setInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.Bucketizer.inputCol" title="pyspark.ml.feature.Bucketizer.inputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">inputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.setOutputCol">
+<code class="descname">setOutputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.setOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.Bucketizer.outputCol" title="pyspark.ml.feature.Bucketizer.outputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.setParams">
+<code class="descname">setParams</code><span class="sig-paren">(</span><em>self</em>, <em>splits=None</em>, <em>inputCol=None</em>, <em>outputCol=None</em>, <em>handleInvalid="error"</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#Bucketizer.setParams"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.setParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets params for this Bucketizer.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.4.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.setSplits">
+<code class="descname">setSplits</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#Bucketizer.setSplits"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.setSplits" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.Bucketizer.splits" title="pyspark.ml.feature.Bucketizer.splits"><code class="xref py py-attr docutils literal notranslate"><span class="pre">splits</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.4.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.Bucketizer.splits">
+<code class="descname">splits</code><em class="property"> = Param(parent='undefined', name='splits', doc='Split points for mapping continuous features into buckets. With n+1 splits, there are n buckets. A bucket defined by splits x,y holds values in the range [x,y) except the last bucket, which also includes y. The splits should be of length >= 3 and strictly increasing. Values at -inf, inf must be explicitly provided to cover all Double values; otherwise, values outside the splits sp [...]
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.transform">
+<code class="descname">transform</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.transform" title="Permalink to this definition">¶</a></dt>
+<dd><p>Transforms the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>transformed dataset</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Bucketizer.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Bucketizer.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.ChiSqSelector">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">ChiSqSelector</code><span class="sig-paren">(</span><em>numTopFeatures=50</em>, <em>featuresCol='features'</em>, <em>outputCol=None</em>, <em>labelCol='label'</em>, <em>selectorType='numTopFeatures'</em>, <em>percentile=0.1</em>, <em>fpr=0.05</em>, <em>fdr=0.05</em>, <em>fwe=0.05</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.ht [...]
+<dd><div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Experimental</p>
+</div>
+<p>Chi-Squared feature selection, which selects categorical features to use for predicting a
+categorical label.
+The selector supports different selection methods: <cite>numTopFeatures</cite>, <cite>percentile</cite>, <cite>fpr</cite>,
+<cite>fdr</cite>, <cite>fwe</cite>.</p>
+<blockquote>
+<div><ul class="simple">
+<li><p><cite>numTopFeatures</cite> chooses a fixed number of top features according to a chi-squared test.</p></li>
+<li><p><cite>percentile</cite> is similar but chooses a fraction of all features
+instead of a fixed number.</p></li>
+<li><p><cite>fpr</cite> chooses all features whose p-values are below a threshold,
+thus controlling the false positive rate of selection.</p></li>
+<li><p><cite>fdr</cite> uses the <a class="reference external" href="https://en.wikipedia.org/wiki/False_discovery_rate#Benjamini.E2.80.93Hochberg_procedure">Benjamini-Hochberg procedure</a>
+to choose all features whose false discovery rate is below a threshold.</p></li>
+<li><p><cite>fwe</cite> chooses all features whose p-values are below a threshold. The threshold is scaled by
+1/numFeatures, thus controlling the family-wise error rate of selection.</p></li>
+</ul>
+</div></blockquote>
+<p>By default, the selection method is <cite>numTopFeatures</cite>, with the default number of top features
+set to 50.</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">pyspark.ml.linalg</span> <span class="k">import</span> <span class="n">Vectors</span>
+<span class="gp">>>> </span><span class="n">df</span> <span class="o">=</span> <span class="n">spark</span><span class="o">.</span><span class="n">createDataFrame</span><span class="p">(</span>
+<span class="gp">... </span> <span class="p">[(</span><span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([</span><span class="mf">0.0</span><span class="p">,</span> <span class="mf">0.0</span><span class="p">,</span> <span class="mf">18.0</span><span class="p">,</span> <span class="mf">1.0</span><span class="p">]),</span> <span class="mf">1.0</span><span class="p">),</span>
+<span class="gp">... </span> <span class="p">(</span><span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([</span><span class="mf">0.0</span><span class="p">,</span> <span class="mf">1.0</span><span class="p">,</span> <span class="mf">12.0</span><span class="p">,</span> <span class="mf">0.0</span><span class="p">]),</span> <span class="mf">0.0</span><span class="p">),</span>
+<span class="gp">... </span> <span class="p">(</span><span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([</span><span class="mf">1.0</span><span class="p">,</span> <span class="mf">0.0</span><span class="p">,</span> <span class="mf">15.0</span><span class="p">,</span> <span class="mf">0.1</span><span class="p">]),</span> <span class="mf">0.0</span><span class="p">)],</span>
+<span class="gp">... </span> <span class="p">[</span><span class="s2">"features"</span><span class="p">,</span> <span class="s2">"label"</span><span class="p">])</span>
+<span class="gp">>>> </span><span class="n">selector</span> <span class="o">=</span> <span class="n">ChiSqSelector</span><span class="p">(</span><span class="n">numTopFeatures</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">outputCol</span><span class="o">=</span><span class="s2">"selectedFeatures"</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">model</span> <span class="o">=</span> <span class="n">selector</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">df</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">selectedFeatures</span>
+<span class="go">DenseVector([18.0])</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">selectedFeatures</span>
+<span class="go">[2]</span>
+<span class="gp">>>> </span><span class="n">chiSqSelectorPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/chi-sq-selector"</span>
+<span class="gp">>>> </span><span class="n">selector</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">chiSqSelectorPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedSelector</span> <span class="o">=</span> <span class="n">ChiSqSelector</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">chiSqSelectorPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedSelector</span><span class="o">.</span><span class="n">getNumTopFeatures</span><span class="p">()</span> <span class="o">==</span> <span class="n">selector</span><span class="o">.</span><span class="n">getNumTopFeatures</span><span class="p">()</span>
+<span class="go">True</span>
+<span class="gp">>>> </span><span class="n">modelPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/chi-sq-selector-model"</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">modelPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedModel</span> <span class="o">=</span> <span class="n">ChiSqSelectorModel</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">modelPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedModel</span><span class="o">.</span><span class="n">selectedFeatures</span> <span class="o">==</span> <span class="n">model</span><span class="o">.</span><span class="n">selectedFeatures</span>
+<span class="go">True</span>
+</pre></div>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.ChiSqSelector.fdr">
+<code class="descname">fdr</code><em class="property"> = Param(parent='undefined', name='fdr', doc='The upper bound of the expected false discovery rate.')</em><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.fdr" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.ChiSqSelector.featuresCol">
+<code class="descname">featuresCol</code><em class="property"> = Param(parent='undefined', name='featuresCol', doc='features column name.')</em><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.featuresCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.fit">
+<code class="descname">fit</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.fit" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params. If a list/tuple of
+param maps is given, this calls fit on each param map and returns a list of
+models.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>fitted model(s)</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.fitMultiple">
+<code class="descname">fitMultiple</code><span class="sig-paren">(</span><em>dataset</em>, <em>paramMaps</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.fitMultiple" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset for each param map in <cite>paramMaps</cite>.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a>.</p></li>
+<li><p><strong>paramMaps</strong> – A Sequence of param maps.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>A thread safe iterable which contains one model for each param map. Each
+call to <cite>next(modelIterator)</cite> will return <cite>(index, model)</cite> where model was fit
+using <cite>paramMaps[index]</cite>. <cite>index</cite> values may not be sequential.</p>
+</dd>
+</dl>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>DeveloperApi</p>
+</div>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Experimental</p>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.ChiSqSelector.fpr">
+<code class="descname">fpr</code><em class="property"> = Param(parent='undefined', name='fpr', doc='The highest p-value for features to be kept.')</em><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.fpr" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.ChiSqSelector.fwe">
+<code class="descname">fwe</code><em class="property"> = Param(parent='undefined', name='fwe', doc='The upper bound of the expected family-wise error rate.')</em><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.fwe" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.getFdr">
+<code class="descname">getFdr</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#ChiSqSelector.getFdr"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.getFdr" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of fdr or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.getFeaturesCol">
+<code class="descname">getFeaturesCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.getFeaturesCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of featuresCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.getFpr">
+<code class="descname">getFpr</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#ChiSqSelector.getFpr"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.getFpr" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of fpr or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.1.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.getFwe">
+<code class="descname">getFwe</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#ChiSqSelector.getFwe"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.getFwe" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of fwe or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.getLabelCol">
+<code class="descname">getLabelCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.getLabelCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of labelCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.getNumTopFeatures">
+<code class="descname">getNumTopFeatures</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#ChiSqSelector.getNumTopFeatures"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.getNumTopFeatures" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of numTopFeatures or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.getOutputCol">
+<code class="descname">getOutputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.getOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of outputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.getPercentile">
+<code class="descname">getPercentile</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#ChiSqSelector.getPercentile"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.getPercentile" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of percentile or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.1.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.getSelectorType">
+<code class="descname">getSelectorType</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#ChiSqSelector.getSelectorType"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.getSelectorType" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of selectorType or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.1.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.ChiSqSelector.labelCol">
+<code class="descname">labelCol</code><em class="property"> = Param(parent='undefined', name='labelCol', doc='label column name.')</em><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.labelCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.ChiSqSelector.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.ChiSqSelector.numTopFeatures">
+<code class="descname">numTopFeatures</code><em class="property"> = Param(parent='undefined', name='numTopFeatures', doc='Number of features that selector will select, ordered by ascending p-value. If the number of features is < numTopFeatures, then this will select all features.')</em><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.numTopFeatures" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.ChiSqSelector.outputCol">
+<code class="descname">outputCol</code><em class="property"> = Param(parent='undefined', name='outputCol', doc='output column name.')</em><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.outputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.ChiSqSelector.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.ChiSqSelector.percentile">
+<code class="descname">percentile</code><em class="property"> = Param(parent='undefined', name='percentile', doc='Percentile of features that selector will select, ordered by ascending p-value.')</em><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.percentile" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.ChiSqSelector.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.ChiSqSelector.selectorType">
+<code class="descname">selectorType</code><em class="property"> = Param(parent='undefined', name='selectorType', doc='The selector type of the ChisqSelector. Supported options: numTopFeatures (default), percentile, fpr, fdr, fwe.')</em><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.selectorType" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.setFdr">
+<code class="descname">setFdr</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#ChiSqSelector.setFdr"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.setFdr" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.ChiSqSelector.fdr" title="pyspark.ml.feature.ChiSqSelector.fdr"><code class="xref py py-attr docutils literal notranslate"><span class="pre">fdr</span></code></a>.
+Only applicable when selectorType = “fdr”.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.setFeaturesCol">
+<code class="descname">setFeaturesCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.setFeaturesCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.ChiSqSelector.featuresCol" title="pyspark.ml.feature.ChiSqSelector.featuresCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">featuresCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.setFpr">
+<code class="descname">setFpr</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#ChiSqSelector.setFpr"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.setFpr" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.ChiSqSelector.fpr" title="pyspark.ml.feature.ChiSqSelector.fpr"><code class="xref py py-attr docutils literal notranslate"><span class="pre">fpr</span></code></a>.
+Only applicable when selectorType = “fpr”.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.1.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.setFwe">
+<code class="descname">setFwe</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#ChiSqSelector.setFwe"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.setFwe" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.ChiSqSelector.fwe" title="pyspark.ml.feature.ChiSqSelector.fwe"><code class="xref py py-attr docutils literal notranslate"><span class="pre">fwe</span></code></a>.
+Only applicable when selectorType = “fwe”.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.setLabelCol">
+<code class="descname">setLabelCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.setLabelCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.ChiSqSelector.labelCol" title="pyspark.ml.feature.ChiSqSelector.labelCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">labelCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.setNumTopFeatures">
+<code class="descname">setNumTopFeatures</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#ChiSqSelector.setNumTopFeatures"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.setNumTopFeatures" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.ChiSqSelector.numTopFeatures" title="pyspark.ml.feature.ChiSqSelector.numTopFeatures"><code class="xref py py-attr docutils literal notranslate"><span class="pre">numTopFeatures</span></code></a>.
+Only applicable when selectorType = “numTopFeatures”.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.setOutputCol">
+<code class="descname">setOutputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.setOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.ChiSqSelector.outputCol" title="pyspark.ml.feature.ChiSqSelector.outputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.setParams">
+<code class="descname">setParams</code><span class="sig-paren">(</span><em>self</em>, <em>numTopFeatures=50</em>, <em>featuresCol="features"</em>, <em>outputCol=None</em>, <em>labelCol="labels"</em>, <em>selectorType="numTopFeatures"</em>, <em>percentile=0.1</em>, <em>fpr=0.05</em>, <em>fdr=0.05</em>, <em>fwe=0.05</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#ChiSqSelector.setParams"><span class="vie [...]
+<dd><p>Sets params for this ChiSqSelector.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.setPercentile">
+<code class="descname">setPercentile</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#ChiSqSelector.setPercentile"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.setPercentile" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.ChiSqSelector.percentile" title="pyspark.ml.feature.ChiSqSelector.percentile"><code class="xref py py-attr docutils literal notranslate"><span class="pre">percentile</span></code></a>.
+Only applicable when selectorType = “percentile”.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.1.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.setSelectorType">
+<code class="descname">setSelectorType</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#ChiSqSelector.setSelectorType"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.setSelectorType" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.ChiSqSelector.selectorType" title="pyspark.ml.feature.ChiSqSelector.selectorType"><code class="xref py py-attr docutils literal notranslate"><span class="pre">selectorType</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.1.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelector.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelector.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.ChiSqSelectorModel">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">ChiSqSelectorModel</code><span class="sig-paren">(</span><em>java_model=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#ChiSqSelectorModel"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelectorModel" title="Permalink to this definition">¶</a></dt>
+<dd><div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Experimental</p>
+</div>
+<p>Model fitted by <a class="reference internal" href="#pyspark.ml.feature.ChiSqSelector" title="pyspark.ml.feature.ChiSqSelector"><code class="xref py py-class docutils literal notranslate"><span class="pre">ChiSqSelector</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelectorModel.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelectorModel.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelectorModel.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelectorModel.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelectorModel.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelectorModel.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelectorModel.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelectorModel.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelectorModel.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelectorModel.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelectorModel.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelectorModel.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelectorModel.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelectorModel.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelectorModel.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelectorModel.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelectorModel.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelectorModel.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelectorModel.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelectorModel.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.ChiSqSelectorModel.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelectorModel.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.ChiSqSelectorModel.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelectorModel.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.ChiSqSelectorModel.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelectorModel.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelectorModel.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelectorModel.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.ChiSqSelectorModel.selectedFeatures">
+<code class="descname">selectedFeatures</code><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelectorModel.selectedFeatures" title="Permalink to this definition">¶</a></dt>
+<dd><p>List of indices to select (filter).</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelectorModel.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelectorModel.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelectorModel.transform">
+<code class="descname">transform</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelectorModel.transform" title="Permalink to this definition">¶</a></dt>
+<dd><p>Transforms the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>transformed dataset</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ChiSqSelectorModel.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ChiSqSelectorModel.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.CountVectorizer">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">CountVectorizer</code><span class="sig-paren">(</span><em>minTF=1.0</em>, <em>minDF=1.0</em>, <em>maxDF=9223372036854775807</em>, <em>vocabSize=262144</em>, <em>binary=False</em>, <em>inputCol=None</em>, <em>outputCol=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#CountVectorizer"><span class="viewcode-link">[source]</ [...]
+<dd><p>Extracts a vocabulary from document collections and generates a <a class="reference internal" href="#pyspark.ml.feature.CountVectorizerModel" title="pyspark.ml.feature.CountVectorizerModel"><code class="xref py py-attr docutils literal notranslate"><span class="pre">CountVectorizerModel</span></code></a>.</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">df</span> <span class="o">=</span> <span class="n">spark</span><span class="o">.</span><span class="n">createDataFrame</span><span class="p">(</span>
+<span class="gp">... </span> <span class="p">[(</span><span class="mi">0</span><span class="p">,</span> <span class="p">[</span><span class="s2">"a"</span><span class="p">,</span> <span class="s2">"b"</span><span class="p">,</span> <span class="s2">"c"</span><span class="p">]),</span> <span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="p">[</span><span class="s2">"a"</span><span class="p">,</span> <span class="s [...]
+<span class="gp">... </span> <span class="p">[</span><span class="s2">"label"</span><span class="p">,</span> <span class="s2">"raw"</span><span class="p">])</span>
+<span class="gp">>>> </span><span class="n">cv</span> <span class="o">=</span> <span class="n">CountVectorizer</span><span class="p">(</span><span class="n">inputCol</span><span class="o">=</span><span class="s2">"raw"</span><span class="p">,</span> <span class="n">outputCol</span><span class="o">=</span><span class="s2">"vectors"</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">model</span> <span class="o">=</span> <span class="n">cv</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">df</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">show</span><span class="p">(</span><span class="n">truncate</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+<span class="go">+-----+---------------+-------------------------+</span>
+<span class="go">|label|raw |vectors |</span>
+<span class="go">+-----+---------------+-------------------------+</span>
+<span class="go">|0 |[a, b, c] |(3,[0,1,2],[1.0,1.0,1.0])|</span>
+<span class="go">|1 |[a, b, b, c, a]|(3,[0,1,2],[2.0,2.0,1.0])|</span>
+<span class="go">+-----+---------------+-------------------------+</span>
+<span class="gp">...</span>
+<span class="gp">>>> </span><span class="nb">sorted</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">vocabulary</span><span class="p">)</span> <span class="o">==</span> <span class="p">[</span><span class="s1">'a'</span><span class="p">,</span> <span class="s1">'b'</span><span class="p">,</span> <span class="s1">'c'</span><span class="p">]</span>
+<span class="go">True</span>
+<span class="gp">>>> </span><span class="n">countVectorizerPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/count-vectorizer"</span>
+<span class="gp">>>> </span><span class="n">cv</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">countVectorizerPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedCv</span> <span class="o">=</span> <span class="n">CountVectorizer</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">countVectorizerPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedCv</span><span class="o">.</span><span class="n">getMinDF</span><span class="p">()</span> <span class="o">==</span> <span class="n">cv</span><span class="o">.</span><span class="n">getMinDF</span><span class="p">()</span>
+<span class="go">True</span>
+<span class="gp">>>> </span><span class="n">loadedCv</span><span class="o">.</span><span class="n">getMinTF</span><span class="p">()</span> <span class="o">==</span> <span class="n">cv</span><span class="o">.</span><span class="n">getMinTF</span><span class="p">()</span>
+<span class="go">True</span>
+<span class="gp">>>> </span><span class="n">loadedCv</span><span class="o">.</span><span class="n">getVocabSize</span><span class="p">()</span> <span class="o">==</span> <span class="n">cv</span><span class="o">.</span><span class="n">getVocabSize</span><span class="p">()</span>
+<span class="go">True</span>
+<span class="gp">>>> </span><span class="n">modelPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/count-vectorizer-model"</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">modelPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedModel</span> <span class="o">=</span> <span class="n">CountVectorizerModel</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">modelPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedModel</span><span class="o">.</span><span class="n">vocabulary</span> <span class="o">==</span> <span class="n">model</span><span class="o">.</span><span class="n">vocabulary</span>
+<span class="go">True</span>
+<span class="gp">>>> </span><span class="n">fromVocabModel</span> <span class="o">=</span> <span class="n">CountVectorizerModel</span><span class="o">.</span><span class="n">from_vocabulary</span><span class="p">([</span><span class="s2">"a"</span><span class="p">,</span> <span class="s2">"b"</span><span class="p">,</span> <span class="s2">"c"</span><span class="p">],</span>
+<span class="gp">... </span> <span class="n">inputCol</span><span class="o">=</span><span class="s2">"raw"</span><span class="p">,</span> <span class="n">outputCol</span><span class="o">=</span><span class="s2">"vectors"</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">fromVocabModel</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">show</span><span class="p">(</span><span class="n">truncate</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+<span class="go">+-----+---------------+-------------------------+</span>
+<span class="go">|label|raw |vectors |</span>
+<span class="go">+-----+---------------+-------------------------+</span>
+<span class="go">|0 |[a, b, c] |(3,[0,1,2],[1.0,1.0,1.0])|</span>
+<span class="go">|1 |[a, b, b, c, a]|(3,[0,1,2],[2.0,2.0,1.0])|</span>
+<span class="go">+-----+---------------+-------------------------+</span>
+<span class="gp">...</span>
+</pre></div>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+<dl class="attribute">
+<dt id="pyspark.ml.feature.CountVectorizer.binary">
+<code class="descname">binary</code><em class="property"> = Param(parent='undefined', name='binary', doc='Binary toggle to control the output vector values. If True, all nonzero counts (after minTF filter applied) are set to 1. This is useful for discrete probabilistic models that model binary events rather than integer counts. Default False')</em><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.binary" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.fit">
+<code class="descname">fit</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.fit" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params. If a list/tuple of
+param maps is given, this calls fit on each param map and returns a list of
+models.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>fitted model(s)</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.fitMultiple">
+<code class="descname">fitMultiple</code><span class="sig-paren">(</span><em>dataset</em>, <em>paramMaps</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.fitMultiple" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset for each param map in <cite>paramMaps</cite>.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a>.</p></li>
+<li><p><strong>paramMaps</strong> – A Sequence of param maps.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>A thread safe iterable which contains one model for each param map. Each
+call to <cite>next(modelIterator)</cite> will return <cite>(index, model)</cite> where model was fit
+using <cite>paramMaps[index]</cite>. <cite>index</cite> values may not be sequential.</p>
+</dd>
+</dl>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>DeveloperApi</p>
+</div>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Experimental</p>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.getBinary">
+<code class="descname">getBinary</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.getBinary" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of binary or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.getInputCol">
+<code class="descname">getInputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.getInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of inputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.getMaxDF">
+<code class="descname">getMaxDF</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.getMaxDF" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of maxDF or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.4.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.getMinDF">
+<code class="descname">getMinDF</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.getMinDF" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of minDF or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.getMinTF">
+<code class="descname">getMinTF</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.getMinTF" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of minTF or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.getOutputCol">
+<code class="descname">getOutputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.getOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of outputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.getVocabSize">
+<code class="descname">getVocabSize</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.getVocabSize" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of vocabSize or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.CountVectorizer.inputCol">
+<code class="descname">inputCol</code><em class="property"> = Param(parent='undefined', name='inputCol', doc='input column name.')</em><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.inputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.CountVectorizer.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.CountVectorizer.maxDF">
+<code class="descname">maxDF</code><em class="property"> = Param(parent='undefined', name='maxDF', doc='Specifies the maximum number of different documents a term could appear in to be included in the vocabulary. A term that appears more than the threshold will be ignored. If this is an integer >= 1, this specifies the maximum number of documents the term could appear in; if this is a double in [0,1), then this specifies the maximum fraction of documents the term could appear in. Defa [...]
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.CountVectorizer.minDF">
+<code class="descname">minDF</code><em class="property"> = Param(parent='undefined', name='minDF', doc='Specifies the minimum number of different documents a term must appear in to be included in the vocabulary. If this is an integer >= 1, this specifies the number of documents the term must appear in; if this is a double in [0,1), then this specifies the fraction of documents. Default 1.0')</em><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.minDF" title="Permalink to [...]
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.CountVectorizer.minTF">
+<code class="descname">minTF</code><em class="property"> = Param(parent='undefined', name='minTF', doc="Filter to ignore rare words in a document. For each document, terms with frequency/count less than the given threshold are ignored. If this is an integer >= 1, then this specifies a count (of times the term must appear in the document); if this is a double in [0,1), then this specifies a fraction (out of the document's token count). Note that the parameter is only used in trans [...]
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.CountVectorizer.outputCol">
+<code class="descname">outputCol</code><em class="property"> = Param(parent='undefined', name='outputCol', doc='output column name.')</em><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.outputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.CountVectorizer.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.CountVectorizer.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.setBinary">
+<code class="descname">setBinary</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#CountVectorizer.setBinary"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.setBinary" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.CountVectorizer.binary" title="pyspark.ml.feature.CountVectorizer.binary"><code class="xref py py-attr docutils literal notranslate"><span class="pre">binary</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.setInputCol">
+<code class="descname">setInputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.setInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.CountVectorizer.inputCol" title="pyspark.ml.feature.CountVectorizer.inputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">inputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.setMaxDF">
+<code class="descname">setMaxDF</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#CountVectorizer.setMaxDF"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.setMaxDF" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.CountVectorizer.maxDF" title="pyspark.ml.feature.CountVectorizer.maxDF"><code class="xref py py-attr docutils literal notranslate"><span class="pre">maxDF</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.4.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.setMinDF">
+<code class="descname">setMinDF</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#CountVectorizer.setMinDF"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.setMinDF" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.CountVectorizer.minDF" title="pyspark.ml.feature.CountVectorizer.minDF"><code class="xref py py-attr docutils literal notranslate"><span class="pre">minDF</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.setMinTF">
+<code class="descname">setMinTF</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#CountVectorizer.setMinTF"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.setMinTF" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.CountVectorizer.minTF" title="pyspark.ml.feature.CountVectorizer.minTF"><code class="xref py py-attr docutils literal notranslate"><span class="pre">minTF</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.setOutputCol">
+<code class="descname">setOutputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.setOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.CountVectorizer.outputCol" title="pyspark.ml.feature.CountVectorizer.outputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.setParams">
+<code class="descname">setParams</code><span class="sig-paren">(</span><em>self</em>, <em>minTF=1.0</em>, <em>minDF=1.0</em>, <em>maxDF=2 ** 63 - 1</em>, <em>vocabSize=1 << 18</em>, <em>binary=False</em>, <em>inputCol=None</em>, <em>outputCol=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#CountVectorizer.setParams"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.CountVector [...]
+<dd><p>Set the params for the CountVectorizer</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.setVocabSize">
+<code class="descname">setVocabSize</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#CountVectorizer.setVocabSize"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.setVocabSize" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.CountVectorizer.vocabSize" title="pyspark.ml.feature.CountVectorizer.vocabSize"><code class="xref py py-attr docutils literal notranslate"><span class="pre">vocabSize</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.CountVectorizer.vocabSize">
+<code class="descname">vocabSize</code><em class="property"> = Param(parent='undefined', name='vocabSize', doc='max size of the vocabulary. Default 1 << 18.')</em><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.vocabSize" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizer.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizer.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.CountVectorizerModel">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">CountVectorizerModel</code><span class="sig-paren">(</span><em>java_model=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#CountVectorizerModel"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel" title="Permalink to this definition">¶</a></dt>
+<dd><p>Model fitted by <a class="reference internal" href="#pyspark.ml.feature.CountVectorizer" title="pyspark.ml.feature.CountVectorizer"><code class="xref py py-class docutils literal notranslate"><span class="pre">CountVectorizer</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+<dl class="attribute">
+<dt id="pyspark.ml.feature.CountVectorizerModel.binary">
+<code class="descname">binary</code><em class="property"> = Param(parent='undefined', name='binary', doc='Binary toggle to control the output vector values. If True, all nonzero counts (after minTF filter applied) are set to 1. This is useful for discrete probabilistic models that model binary events rather than integer counts. Default False')</em><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.binary" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.CountVectorizerModel.from_vocabulary">
+<em class="property">classmethod </em><code class="descname">from_vocabulary</code><span class="sig-paren">(</span><em>vocabulary</em>, <em>inputCol</em>, <em>outputCol=None</em>, <em>minTF=None</em>, <em>binary=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#CountVectorizerModel.from_vocabulary"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.from_vocabulary" [...]
+<dd><p>Construct the model directly from a vocabulary list of strings,
+requires an active SparkContext.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.4.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.getBinary">
+<code class="descname">getBinary</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.getBinary" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of binary or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.getInputCol">
+<code class="descname">getInputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.getInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of inputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.getMaxDF">
+<code class="descname">getMaxDF</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.getMaxDF" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of maxDF or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.4.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.getMinDF">
+<code class="descname">getMinDF</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.getMinDF" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of minDF or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.getMinTF">
+<code class="descname">getMinTF</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.getMinTF" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of minTF or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.getOutputCol">
+<code class="descname">getOutputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.getOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of outputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.getVocabSize">
+<code class="descname">getVocabSize</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.getVocabSize" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of vocabSize or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.CountVectorizerModel.inputCol">
+<code class="descname">inputCol</code><em class="property"> = Param(parent='undefined', name='inputCol', doc='input column name.')</em><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.inputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.CountVectorizerModel.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.CountVectorizerModel.maxDF">
+<code class="descname">maxDF</code><em class="property"> = Param(parent='undefined', name='maxDF', doc='Specifies the maximum number of different documents a term could appear in to be included in the vocabulary. A term that appears more than the threshold will be ignored. If this is an integer >= 1, this specifies the maximum number of documents the term could appear in; if this is a double in [0,1), then this specifies the maximum fraction of documents the term could appear in. Defa [...]
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.CountVectorizerModel.minDF">
+<code class="descname">minDF</code><em class="property"> = Param(parent='undefined', name='minDF', doc='Specifies the minimum number of different documents a term must appear in to be included in the vocabulary. If this is an integer >= 1, this specifies the number of documents the term must appear in; if this is a double in [0,1), then this specifies the fraction of documents. Default 1.0')</em><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.minDF" title="Permali [...]
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.CountVectorizerModel.minTF">
+<code class="descname">minTF</code><em class="property"> = Param(parent='undefined', name='minTF', doc="Filter to ignore rare words in a document. For each document, terms with frequency/count less than the given threshold are ignored. If this is an integer >= 1, then this specifies a count (of times the term must appear in the document); if this is a double in [0,1), then this specifies a fraction (out of the document's token count). Note that the parameter is only used in trans [...]
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.CountVectorizerModel.outputCol">
+<code class="descname">outputCol</code><em class="property"> = Param(parent='undefined', name='outputCol', doc='output column name.')</em><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.outputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.CountVectorizerModel.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.CountVectorizerModel.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.setBinary">
+<code class="descname">setBinary</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#CountVectorizerModel.setBinary"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.setBinary" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.CountVectorizerModel.binary" title="pyspark.ml.feature.CountVectorizerModel.binary"><code class="xref py py-attr docutils literal notranslate"><span class="pre">binary</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.4.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.setInputCol">
+<code class="descname">setInputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.setInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.CountVectorizerModel.inputCol" title="pyspark.ml.feature.CountVectorizerModel.inputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">inputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.setMinTF">
+<code class="descname">setMinTF</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#CountVectorizerModel.setMinTF"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.setMinTF" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.CountVectorizerModel.minTF" title="pyspark.ml.feature.CountVectorizerModel.minTF"><code class="xref py py-attr docutils literal notranslate"><span class="pre">minTF</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.4.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.setOutputCol">
+<code class="descname">setOutputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.setOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.CountVectorizerModel.outputCol" title="pyspark.ml.feature.CountVectorizerModel.outputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.transform">
+<code class="descname">transform</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.transform" title="Permalink to this definition">¶</a></dt>
+<dd><p>Transforms the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>transformed dataset</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.CountVectorizerModel.vocabSize">
+<code class="descname">vocabSize</code><em class="property"> = Param(parent='undefined', name='vocabSize', doc='max size of the vocabulary. Default 1 << 18.')</em><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.vocabSize" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.CountVectorizerModel.vocabulary">
+<code class="descname">vocabulary</code><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.vocabulary" title="Permalink to this definition">¶</a></dt>
+<dd><p>An array of terms in the vocabulary.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.CountVectorizerModel.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.CountVectorizerModel.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.DCT">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">DCT</code><span class="sig-paren">(</span><em>inverse=False</em>, <em>inputCol=None</em>, <em>outputCol=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#DCT"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.DCT" title="Permalink to this definition">¶</a></dt>
+<dd><p>A feature transformer that takes the 1D discrete cosine transform
+of a real vector. No zero padding is performed on the input vector.
+It returns a real vector of the same length representing the DCT.
+The return vector is scaled such that the transform matrix is
+unitary (aka scaled DCT-II).</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
+<p><a class="reference external" href="https://en.wikipedia.org/wiki/Discrete_cosine_transform#DCT-IIWikipedia">More information on Wikipedia</a>.</p>
+</div>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">pyspark.ml.linalg</span> <span class="k">import</span> <span class="n">Vectors</span>
+<span class="gp">>>> </span><span class="n">df1</span> <span class="o">=</span> <span class="n">spark</span><span class="o">.</span><span class="n">createDataFrame</span><span class="p">([(</span><span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([</span><span class="mf">5.0</span><span class="p">,</span> <span class="mf">8.0</span><span class="p">,</span> <span class="mf">6.0</span><span class="p">]),)],</span> <span class="p">[</ [...]
+<span class="gp">>>> </span><span class="n">dct</span> <span class="o">=</span> <span class="n">DCT</span><span class="p">(</span><span class="n">inverse</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">inputCol</span><span class="o">=</span><span class="s2">"vec"</span><span class="p">,</span> <span class="n">outputCol</span><span class="o">=</span><span class="s2">"resultVec"</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">df2</span> <span class="o">=</span> <span class="n">dct</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df1</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">df2</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">resultVec</span>
+<span class="go">DenseVector([10.969..., -0.707..., -2.041...])</span>
+<span class="gp">>>> </span><span class="n">df3</span> <span class="o">=</span> <span class="n">DCT</span><span class="p">(</span><span class="n">inverse</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">inputCol</span><span class="o">=</span><span class="s2">"resultVec"</span><span class="p">,</span> <span class="n">outputCol</span><span class="o">=</span><span class="s2">"origVec"</span><span class="p">)</spa [...]
+<span class="gp">>>> </span><span class="n">df3</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">origVec</span>
+<span class="go">DenseVector([5.0, 8.0, 6.0])</span>
+<span class="gp">>>> </span><span class="n">dctPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/dct"</span>
+<span class="gp">>>> </span><span class="n">dct</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">dctPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedDtc</span> <span class="o">=</span> <span class="n">DCT</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">dctPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedDtc</span><span class="o">.</span><span class="n">getInverse</span><span class="p">()</span>
+<span class="go">False</span>
+</pre></div>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.getInputCol">
+<code class="descname">getInputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.getInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of inputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.getInverse">
+<code class="descname">getInverse</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#DCT.getInverse"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.DCT.getInverse" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of inverse or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.getOutputCol">
+<code class="descname">getOutputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.getOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of outputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.DCT.inputCol">
+<code class="descname">inputCol</code><em class="property"> = Param(parent='undefined', name='inputCol', doc='input column name.')</em><a class="headerlink" href="#pyspark.ml.feature.DCT.inputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.DCT.inverse">
+<code class="descname">inverse</code><em class="property"> = Param(parent='undefined', name='inverse', doc='Set transformer to perform inverse DCT, default False.')</em><a class="headerlink" href="#pyspark.ml.feature.DCT.inverse" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.DCT.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.DCT.outputCol">
+<code class="descname">outputCol</code><em class="property"> = Param(parent='undefined', name='outputCol', doc='output column name.')</em><a class="headerlink" href="#pyspark.ml.feature.DCT.outputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.DCT.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.DCT.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.DCT.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.setInputCol">
+<code class="descname">setInputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.setInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.DCT.inputCol" title="pyspark.ml.feature.DCT.inputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">inputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.setInverse">
+<code class="descname">setInverse</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#DCT.setInverse"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.DCT.setInverse" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.DCT.inverse" title="pyspark.ml.feature.DCT.inverse"><code class="xref py py-attr docutils literal notranslate"><span class="pre">inverse</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.setOutputCol">
+<code class="descname">setOutputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.setOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.DCT.outputCol" title="pyspark.ml.feature.DCT.outputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.setParams">
+<code class="descname">setParams</code><span class="sig-paren">(</span><em>self</em>, <em>inverse=False</em>, <em>inputCol=None</em>, <em>outputCol=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#DCT.setParams"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.DCT.setParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets params for this DCT.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.transform">
+<code class="descname">transform</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.transform" title="Permalink to this definition">¶</a></dt>
+<dd><p>Transforms the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>transformed dataset</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.DCT.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.DCT.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.ElementwiseProduct">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">ElementwiseProduct</code><span class="sig-paren">(</span><em>scalingVec=None</em>, <em>inputCol=None</em>, <em>outputCol=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#ElementwiseProduct"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct" title="Permalink [...]
+<dd><p>Outputs the Hadamard product (i.e., the element-wise product) of each input vector
+with a provided “weight” vector. In other words, it scales each column of the dataset
+by a scalar multiplier.</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">pyspark.ml.linalg</span> <span class="k">import</span> <span class="n">Vectors</span>
+<span class="gp">>>> </span><span class="n">df</span> <span class="o">=</span> <span class="n">spark</span><span class="o">.</span><span class="n">createDataFrame</span><span class="p">([(</span><span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([</span><span class="mf">2.0</span><span class="p">,</span> <span class="mf">1.0</span><span class="p">,</span> <span class="mf">3.0</span><span class="p">]),)],</span> <span class="p">[</s [...]
+<span class="gp">>>> </span><span class="n">ep</span> <span class="o">=</span> <span class="n">ElementwiseProduct</span><span class="p">(</span><span class="n">scalingVec</span><span class="o">=</span><span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([</span><span class="mf">1.0</span><span class="p">,</span> <span class="mf">2.0</span><span class="p">,</span> <span class="mf">3.0</span><span class="p">]),</span>
+<span class="gp">... </span> <span class="n">inputCol</span><span class="o">=</span><span class="s2">"values"</span><span class="p">,</span> <span class="n">outputCol</span><span class="o">=</span><span class="s2">"eprod"</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">ep</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">eprod</span>
+<span class="go">DenseVector([2.0, 2.0, 9.0])</span>
+<span class="gp">>>> </span><span class="n">ep</span><span class="o">.</span><span class="n">setParams</span><span class="p">(</span><span class="n">scalingVec</span><span class="o">=</span><span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([</span><span class="mf">2.0</span><span class="p">,</span> <span class="mf">3.0</span><span class="p">,</span> <span class="mf">5.0</span><span class="p">]))</span><span class="o">.</span><span [...]
+<span class="go">DenseVector([4.0, 3.0, 15.0])</span>
+<span class="gp">>>> </span><span class="n">elementwiseProductPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/elementwise-product"</span>
+<span class="gp">>>> </span><span class="n">ep</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">elementwiseProductPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedEp</span> <span class="o">=</span> <span class="n">ElementwiseProduct</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">elementwiseProductPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedEp</span><span class="o">.</span><span class="n">getScalingVec</span><span class="p">()</span> <span class="o">==</span> <span class="n">ep</span><span class="o">.</span><span class="n">getScalingVec</span><span class="p">()</span>
+<span class="go">True</span>
+</pre></div>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.5.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.getInputCol">
+<code class="descname">getInputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.getInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of inputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.getOutputCol">
+<code class="descname">getOutputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.getOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of outputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.getScalingVec">
+<code class="descname">getScalingVec</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#ElementwiseProduct.getScalingVec"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.getScalingVec" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of scalingVec or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.ElementwiseProduct.inputCol">
+<code class="descname">inputCol</code><em class="property"> = Param(parent='undefined', name='inputCol', doc='input column name.')</em><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.inputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.ElementwiseProduct.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.ElementwiseProduct.outputCol">
+<code class="descname">outputCol</code><em class="property"> = Param(parent='undefined', name='outputCol', doc='output column name.')</em><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.outputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.ElementwiseProduct.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.ElementwiseProduct.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.ElementwiseProduct.scalingVec">
+<code class="descname">scalingVec</code><em class="property"> = Param(parent='undefined', name='scalingVec', doc='Vector for hadamard product.')</em><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.scalingVec" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.setInputCol">
+<code class="descname">setInputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.setInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.ElementwiseProduct.inputCol" title="pyspark.ml.feature.ElementwiseProduct.inputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">inputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.setOutputCol">
+<code class="descname">setOutputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.setOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.ElementwiseProduct.outputCol" title="pyspark.ml.feature.ElementwiseProduct.outputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.setParams">
+<code class="descname">setParams</code><span class="sig-paren">(</span><em>self</em>, <em>scalingVec=None</em>, <em>inputCol=None</em>, <em>outputCol=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#ElementwiseProduct.setParams"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.setParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets params for this ElementwiseProduct.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.5.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.setScalingVec">
+<code class="descname">setScalingVec</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#ElementwiseProduct.setScalingVec"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.setScalingVec" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.ElementwiseProduct.scalingVec" title="pyspark.ml.feature.ElementwiseProduct.scalingVec"><code class="xref py py-attr docutils literal notranslate"><span class="pre">scalingVec</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.transform">
+<code class="descname">transform</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.transform" title="Permalink to this definition">¶</a></dt>
+<dd><p>Transforms the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>transformed dataset</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ElementwiseProduct.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ElementwiseProduct.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.FeatureHasher">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">FeatureHasher</code><span class="sig-paren">(</span><em>numFeatures=262144</em>, <em>inputCols=None</em>, <em>outputCol=None</em>, <em>categoricalCols=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#FeatureHasher"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.FeatureHashe [...]
+<dd><div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Experimental</p>
+</div>
+<p>Feature hashing projects a set of categorical or numerical features into a feature vector of
+specified dimension (typically substantially smaller than that of the original feature
+space). This is done using the hashing trick (<a class="reference external" href="https://en.wikipedia.org/wiki/Feature_hashing">https://en.wikipedia.org/wiki/Feature_hashing</a>)
+to map features to indices in the feature vector.</p>
+<p>The FeatureHasher transformer operates on multiple columns. Each column may contain either
+numeric or categorical features. Behavior and handling of column data types is as follows:</p>
+<ul class="simple">
+<li><dl class="simple">
+<dt>Numeric columns:</dt><dd><p>For numeric features, the hash value of the column name is used to map the
+feature value to its index in the feature vector. By default, numeric features
+are not treated as categorical (even when they are integers). To treat them
+as categorical, specify the relevant columns in <cite>categoricalCols</cite>.</p>
+</dd>
+</dl>
+</li>
+<li><dl class="simple">
+<dt>String columns:</dt><dd><p>For categorical features, the hash value of the string “column_name=value”
+is used to map to the vector index, with an indicator value of <cite>1.0</cite>.
+Thus, categorical features are “one-hot” encoded
+(similarly to using <a class="reference internal" href="#pyspark.ml.feature.OneHotEncoder" title="pyspark.ml.feature.OneHotEncoder"><code class="xref py py-class docutils literal notranslate"><span class="pre">OneHotEncoder</span></code></a> with <cite>dropLast=false</cite>).</p>
+</dd>
+</dl>
+</li>
+<li><dl class="simple">
+<dt>Boolean columns:</dt><dd><p>Boolean values are treated in the same way as string columns. That is,
+boolean features are represented as “column_name=true” or “column_name=false”,
+with an indicator value of <cite>1.0</cite>.</p>
+</dd>
+</dl>
+</li>
+</ul>
+<p>Null (missing) values are ignored (implicitly zero in the resulting feature vector).</p>
+<p>Since a simple modulo is used to transform the hash function to a vector index,
+it is advisable to use a power of two as the <cite>numFeatures</cite> parameter;
+otherwise the features will not be mapped evenly to the vector indices.</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">data</span> <span class="o">=</span> <span class="p">[(</span><span class="mf">2.0</span><span class="p">,</span> <span class="kc">True</span><span class="p">,</span> <span class="s2">"1"</span><span class="p">,</span> <span class="s2">"foo"</span><span class="p">),</span> <span class="p">(</span><span class="mf">3.0</span><span clas [...]
+<span class="gp">>>> </span><span class="n">cols</span> <span class="o">=</span> <span class="p">[</span><span class="s2">"real"</span><span class="p">,</span> <span class="s2">"bool"</span><span class="p">,</span> <span class="s2">"stringNum"</span><span class="p">,</span> <span class="s2">"string"</span><span class="p">]</span>
+<span class="gp">>>> </span><span class="n">df</span> <span class="o">=</span> <span class="n">spark</span><span class="o">.</span><span class="n">createDataFrame</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">cols</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">hasher</span> <span class="o">=</span> <span class="n">FeatureHasher</span><span class="p">(</span><span class="n">inputCols</span><span class="o">=</span><span class="n">cols</span><span class="p">,</span> <span class="n">outputCol</span><span class="o">=</span><span class="s2">"features"</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">hasher</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">features</span>
+<span class="go">SparseVector(262144, {174475: 2.0, 247670: 1.0, 257907: 1.0, 262126: 1.0})</span>
+<span class="gp">>>> </span><span class="n">hasher</span><span class="o">.</span><span class="n">setCategoricalCols</span><span class="p">([</span><span class="s2">"real"</span><span class="p">])</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">features</span>
+<span class="go">SparseVector(262144, {171257: 1.0, 247670: 1.0, 257907: 1.0, 262126: 1.0})</span>
+<span class="gp">>>> </span><span class="n">hasherPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/hasher"</span>
+<span class="gp">>>> </span><span class="n">hasher</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">hasherPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedHasher</span> <span class="o">=</span> <span class="n">FeatureHasher</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">hasherPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedHasher</span><span class="o">.</span><span class="n">getNumFeatures</span><span class="p">()</span> <span class="o">==</span> <span class="n">hasher</span><span class="o">.</span><span class="n">getNumFeatures</span><span class="p">()</span>
+<span class="go">True</span>
+<span class="gp">>>> </span><span class="n">loadedHasher</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">features</span> <span class="o">==</span> <span class="n">hasher</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span [...]
+<span class="go">True</span>
+</pre></div>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.3.0.</span></p>
+</div>
+<dl class="attribute">
+<dt id="pyspark.ml.feature.FeatureHasher.categoricalCols">
+<code class="descname">categoricalCols</code><em class="property"> = Param(parent='undefined', name='categoricalCols', doc='numeric columns to treat as categorical')</em><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.categoricalCols" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.getCategoricalCols">
+<code class="descname">getCategoricalCols</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#FeatureHasher.getCategoricalCols"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.getCategoricalCols" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of binary or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.getInputCols">
+<code class="descname">getInputCols</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.getInputCols" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of inputCols or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.getNumFeatures">
+<code class="descname">getNumFeatures</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.getNumFeatures" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of numFeatures or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.getOutputCol">
+<code class="descname">getOutputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.getOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of outputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.FeatureHasher.inputCols">
+<code class="descname">inputCols</code><em class="property"> = Param(parent='undefined', name='inputCols', doc='input column names.')</em><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.inputCols" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.FeatureHasher.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.FeatureHasher.numFeatures">
+<code class="descname">numFeatures</code><em class="property"> = Param(parent='undefined', name='numFeatures', doc='number of features.')</em><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.numFeatures" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.FeatureHasher.outputCol">
+<code class="descname">outputCol</code><em class="property"> = Param(parent='undefined', name='outputCol', doc='output column name.')</em><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.outputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.FeatureHasher.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.FeatureHasher.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.setCategoricalCols">
+<code class="descname">setCategoricalCols</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#FeatureHasher.setCategoricalCols"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.setCategoricalCols" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.FeatureHasher.categoricalCols" title="pyspark.ml.feature.FeatureHasher.categoricalCols"><code class="xref py py-attr docutils literal notranslate"><span class="pre">categoricalCols</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.setInputCols">
+<code class="descname">setInputCols</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.setInputCols" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.FeatureHasher.inputCols" title="pyspark.ml.feature.FeatureHasher.inputCols"><code class="xref py py-attr docutils literal notranslate"><span class="pre">inputCols</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.setNumFeatures">
+<code class="descname">setNumFeatures</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.setNumFeatures" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.FeatureHasher.numFeatures" title="pyspark.ml.feature.FeatureHasher.numFeatures"><code class="xref py py-attr docutils literal notranslate"><span class="pre">numFeatures</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.setOutputCol">
+<code class="descname">setOutputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.setOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.FeatureHasher.outputCol" title="pyspark.ml.feature.FeatureHasher.outputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.setParams">
+<code class="descname">setParams</code><span class="sig-paren">(</span><em>self</em>, <em>numFeatures=1 << 18</em>, <em>inputCols=None</em>, <em>outputCol=None</em>, <em>categoricalCols=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#FeatureHasher.setParams"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.setParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets params for this FeatureHasher.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.transform">
+<code class="descname">transform</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.transform" title="Permalink to this definition">¶</a></dt>
+<dd><p>Transforms the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>transformed dataset</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.FeatureHasher.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.FeatureHasher.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.HashingTF">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">HashingTF</code><span class="sig-paren">(</span><em>numFeatures=262144</em>, <em>binary=False</em>, <em>inputCol=None</em>, <em>outputCol=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#HashingTF"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.HashingTF" title="Permalink t [...]
+<dd><p>Maps a sequence of terms to their term frequencies using the hashing trick.
+Currently we use Austin Appleby’s MurmurHash 3 algorithm (MurmurHash3_x86_32)
+to calculate the hash code value for the term object.
+Since a simple modulo is used to transform the hash function to a column index,
+it is advisable to use a power of two as the numFeatures parameter;
+otherwise the features will not be mapped evenly to the columns.</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">df</span> <span class="o">=</span> <span class="n">spark</span><span class="o">.</span><span class="n">createDataFrame</span><span class="p">([([</span><span class="s2">"a"</span><span class="p">,</span> <span class="s2">"b"</span><span class="p">,</span> <span class="s2">"c"</span><span class="p">],)],</span> <span class=" [...]
+<span class="gp">>>> </span><span class="n">hashingTF</span> <span class="o">=</span> <span class="n">HashingTF</span><span class="p">(</span><span class="n">numFeatures</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">inputCol</span><span class="o">=</span><span class="s2">"words"</span><span class="p">,</span> <span class="n">outputCol</span><span class="o">=</span><span class="s2">"features"</span><span class [...]
+<span class="gp">>>> </span><span class="n">hashingTF</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">features</span>
+<span class="go">SparseVector(10, {0: 1.0, 1: 1.0, 2: 1.0})</span>
+<span class="gp">>>> </span><span class="n">hashingTF</span><span class="o">.</span><span class="n">setParams</span><span class="p">(</span><span class="n">outputCol</span><span class="o">=</span><span class="s2">"freqs"</span><span class="p">)</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">. [...]
+<span class="go">SparseVector(10, {0: 1.0, 1: 1.0, 2: 1.0})</span>
+<span class="gp">>>> </span><span class="n">params</span> <span class="o">=</span> <span class="p">{</span><span class="n">hashingTF</span><span class="o">.</span><span class="n">numFeatures</span><span class="p">:</span> <span class="mi">5</span><span class="p">,</span> <span class="n">hashingTF</span><span class="o">.</span><span class="n">outputCol</span><span class="p">:</span> <span class="s2">"vector"</span><span class="p">}</span>
+<span class="gp">>>> </span><span class="n">hashingTF</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">,</span> <span class="n">params</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">vector</span>
+<span class="go">SparseVector(5, {0: 1.0, 1: 1.0, 2: 1.0})</span>
+<span class="gp">>>> </span><span class="n">hashingTFPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/hashing-tf"</span>
+<span class="gp">>>> </span><span class="n">hashingTF</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">hashingTFPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedHashingTF</span> <span class="o">=</span> <span class="n">HashingTF</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">hashingTFPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedHashingTF</span><span class="o">.</span><span class="n">getNumFeatures</span><span class="p">()</span> <span class="o">==</span> <span class="n">hashingTF</span><span class="o">.</span><span class="n">getNumFeatures</span><span class="p">()</span>
+<span class="go">True</span>
+</pre></div>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+<dl class="attribute">
+<dt id="pyspark.ml.feature.HashingTF.binary">
+<code class="descname">binary</code><em class="property"> = Param(parent='undefined', name='binary', doc='If True, all non zero counts are set to 1. This is useful for discrete probabilistic models that model binary events rather than integer counts. Default False.')</em><a class="headerlink" href="#pyspark.ml.feature.HashingTF.binary" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.getBinary">
+<code class="descname">getBinary</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#HashingTF.getBinary"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.HashingTF.getBinary" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of binary or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.getInputCol">
+<code class="descname">getInputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.getInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of inputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.getNumFeatures">
+<code class="descname">getNumFeatures</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.getNumFeatures" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of numFeatures or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.getOutputCol">
+<code class="descname">getOutputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.getOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of outputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.HashingTF.inputCol">
+<code class="descname">inputCol</code><em class="property"> = Param(parent='undefined', name='inputCol', doc='input column name.')</em><a class="headerlink" href="#pyspark.ml.feature.HashingTF.inputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.HashingTF.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.HashingTF.numFeatures">
+<code class="descname">numFeatures</code><em class="property"> = Param(parent='undefined', name='numFeatures', doc='number of features.')</em><a class="headerlink" href="#pyspark.ml.feature.HashingTF.numFeatures" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.HashingTF.outputCol">
+<code class="descname">outputCol</code><em class="property"> = Param(parent='undefined', name='outputCol', doc='output column name.')</em><a class="headerlink" href="#pyspark.ml.feature.HashingTF.outputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.HashingTF.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.HashingTF.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.HashingTF.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.setBinary">
+<code class="descname">setBinary</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#HashingTF.setBinary"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.HashingTF.setBinary" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.HashingTF.binary" title="pyspark.ml.feature.HashingTF.binary"><code class="xref py py-attr docutils literal notranslate"><span class="pre">binary</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.setInputCol">
+<code class="descname">setInputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.setInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.HashingTF.inputCol" title="pyspark.ml.feature.HashingTF.inputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">inputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.setNumFeatures">
+<code class="descname">setNumFeatures</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.setNumFeatures" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.HashingTF.numFeatures" title="pyspark.ml.feature.HashingTF.numFeatures"><code class="xref py py-attr docutils literal notranslate"><span class="pre">numFeatures</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.setOutputCol">
+<code class="descname">setOutputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.setOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.HashingTF.outputCol" title="pyspark.ml.feature.HashingTF.outputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.setParams">
+<code class="descname">setParams</code><span class="sig-paren">(</span><em>self</em>, <em>numFeatures=1 << 18</em>, <em>binary=False</em>, <em>inputCol=None</em>, <em>outputCol=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#HashingTF.setParams"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.HashingTF.setParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets params for this HashingTF.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.transform">
+<code class="descname">transform</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.transform" title="Permalink to this definition">¶</a></dt>
+<dd><p>Transforms the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>transformed dataset</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.HashingTF.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.HashingTF.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.IDF">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">IDF</code><span class="sig-paren">(</span><em>minDocFreq=0</em>, <em>inputCol=None</em>, <em>outputCol=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#IDF"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.IDF" title="Permalink to this definition">¶</a></dt>
+<dd><p>Compute the Inverse Document Frequency (IDF) given a collection of documents.</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">pyspark.ml.linalg</span> <span class="k">import</span> <span class="n">DenseVector</span>
+<span class="gp">>>> </span><span class="n">df</span> <span class="o">=</span> <span class="n">spark</span><span class="o">.</span><span class="n">createDataFrame</span><span class="p">([(</span><span class="n">DenseVector</span><span class="p">([</span><span class="mf">1.0</span><span class="p">,</span> <span class="mf">2.0</span><span class="p">]),),</span>
+<span class="gp">... </span> <span class="p">(</span><span class="n">DenseVector</span><span class="p">([</span><span class="mf">0.0</span><span class="p">,</span> <span class="mf">1.0</span><span class="p">]),),</span> <span class="p">(</span><span class="n">DenseVector</span><span class="p">([</span><span class="mf">3.0</span><span class="p">,</span> <span class="mf">0.2</span><span class="p">]),)],</span> <span class="p">[</span><span class="s2">"tf"</span><span class="p" [...]
+<span class="gp">>>> </span><span class="n">idf</span> <span class="o">=</span> <span class="n">IDF</span><span class="p">(</span><span class="n">minDocFreq</span><span class="o">=</span><span class="mi">3</span><span class="p">,</span> <span class="n">inputCol</span><span class="o">=</span><span class="s2">"tf"</span><span class="p">,</span> <span class="n">outputCol</span><span class="o">=</span><span class="s2">"idf"</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">model</span> <span class="o">=</span> <span class="n">idf</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">df</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">idf</span>
+<span class="go">DenseVector([0.0, 0.0])</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">idf</span>
+<span class="go">DenseVector([0.0, 0.0])</span>
+<span class="gp">>>> </span><span class="n">idf</span><span class="o">.</span><span class="n">setParams</span><span class="p">(</span><span class="n">outputCol</span><span class="o">=</span><span class="s2">"freqs"</span><span class="p">)</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span> [...]
+<span class="go">DenseVector([0.0, 0.0])</span>
+<span class="gp">>>> </span><span class="n">params</span> <span class="o">=</span> <span class="p">{</span><span class="n">idf</span><span class="o">.</span><span class="n">minDocFreq</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span> <span class="n">idf</span><span class="o">.</span><span class="n">outputCol</span><span class="p">:</span> <span class="s2">"vector"</span><span class="p">}</span>
+<span class="gp">>>> </span><span class="n">idf</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">df</span><span class="p">,</span> <span class="n">params</span><span class="p">)</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">vector</span>
+<span class="go">DenseVector([0.2877, 0.0])</span>
+<span class="gp">>>> </span><span class="n">idfPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/idf"</span>
+<span class="gp">>>> </span><span class="n">idf</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">idfPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedIdf</span> <span class="o">=</span> <span class="n">IDF</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">idfPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedIdf</span><span class="o">.</span><span class="n">getMinDocFreq</span><span class="p">()</span> <span class="o">==</span> <span class="n">idf</span><span class="o">.</span><span class="n">getMinDocFreq</span><span class="p">()</span>
+<span class="go">True</span>
+<span class="gp">>>> </span><span class="n">modelPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/idf-model"</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">modelPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedModel</span> <span class="o">=</span> <span class="n">IDFModel</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">modelPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedModel</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">idf</span> <span class="o">==</span> <span class="n">model</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span [...]
+<span class="go">True</span>
+</pre></div>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.4.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.fit">
+<code class="descname">fit</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.fit" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params. If a list/tuple of
+param maps is given, this calls fit on each param map and returns a list of
+models.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>fitted model(s)</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.fitMultiple">
+<code class="descname">fitMultiple</code><span class="sig-paren">(</span><em>dataset</em>, <em>paramMaps</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.fitMultiple" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset for each param map in <cite>paramMaps</cite>.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a>.</p></li>
+<li><p><strong>paramMaps</strong> – A Sequence of param maps.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>A thread safe iterable which contains one model for each param map. Each
+call to <cite>next(modelIterator)</cite> will return <cite>(index, model)</cite> where model was fit
+using <cite>paramMaps[index]</cite>. <cite>index</cite> values may not be sequential.</p>
+</dd>
+</dl>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>DeveloperApi</p>
+</div>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Experimental</p>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.getInputCol">
+<code class="descname">getInputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.getInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of inputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.getMinDocFreq">
+<code class="descname">getMinDocFreq</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#IDF.getMinDocFreq"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.IDF.getMinDocFreq" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of minDocFreq or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.4.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.getOutputCol">
+<code class="descname">getOutputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.getOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of outputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.IDF.inputCol">
+<code class="descname">inputCol</code><em class="property"> = Param(parent='undefined', name='inputCol', doc='input column name.')</em><a class="headerlink" href="#pyspark.ml.feature.IDF.inputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.IDF.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.IDF.minDocFreq">
+<code class="descname">minDocFreq</code><em class="property"> = Param(parent='undefined', name='minDocFreq', doc='minimum number of documents in which a term should appear for filtering')</em><a class="headerlink" href="#pyspark.ml.feature.IDF.minDocFreq" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.IDF.outputCol">
+<code class="descname">outputCol</code><em class="property"> = Param(parent='undefined', name='outputCol', doc='output column name.')</em><a class="headerlink" href="#pyspark.ml.feature.IDF.outputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.IDF.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.IDF.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.IDF.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.setInputCol">
+<code class="descname">setInputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.setInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.IDF.inputCol" title="pyspark.ml.feature.IDF.inputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">inputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.setMinDocFreq">
+<code class="descname">setMinDocFreq</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#IDF.setMinDocFreq"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.IDF.setMinDocFreq" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.IDF.minDocFreq" title="pyspark.ml.feature.IDF.minDocFreq"><code class="xref py py-attr docutils literal notranslate"><span class="pre">minDocFreq</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.4.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.setOutputCol">
+<code class="descname">setOutputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.setOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.IDF.outputCol" title="pyspark.ml.feature.IDF.outputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.setParams">
+<code class="descname">setParams</code><span class="sig-paren">(</span><em>self</em>, <em>minDocFreq=0</em>, <em>inputCol=None</em>, <em>outputCol=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#IDF.setParams"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.IDF.setParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets params for this IDF.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.4.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDF.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDF.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.IDFModel">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">IDFModel</code><span class="sig-paren">(</span><em>java_model=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#IDFModel"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.IDFModel" title="Permalink to this definition">¶</a></dt>
+<dd><p>Model fitted by <a class="reference internal" href="#pyspark.ml.feature.IDF" title="pyspark.ml.feature.IDF"><code class="xref py py-class docutils literal notranslate"><span class="pre">IDF</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.4.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.feature.IDFModel.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDFModel.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDFModel.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDFModel.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDFModel.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDFModel.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDFModel.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDFModel.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDFModel.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDFModel.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDFModel.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDFModel.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDFModel.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDFModel.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDFModel.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDFModel.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.IDFModel.idf">
+<code class="descname">idf</code><a class="headerlink" href="#pyspark.ml.feature.IDFModel.idf" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the IDF vector.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDFModel.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDFModel.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDFModel.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDFModel.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.IDFModel.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDFModel.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.IDFModel.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.IDFModel.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.IDFModel.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDFModel.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDFModel.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDFModel.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDFModel.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDFModel.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDFModel.transform">
+<code class="descname">transform</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDFModel.transform" title="Permalink to this definition">¶</a></dt>
+<dd><p>Transforms the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>transformed dataset</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IDFModel.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IDFModel.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.Imputer">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">Imputer</code><span class="sig-paren">(</span><em>strategy='mean'</em>, <em>missingValue=nan</em>, <em>inputCols=None</em>, <em>outputCols=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#Imputer"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.Imputer" title="Permalink to t [...]
+<dd><div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Experimental</p>
+</div>
+<p>Imputation estimator for completing missing values, either using the mean or the median
+of the columns in which the missing values are located. The input columns should be of
+DoubleType or FloatType. Currently Imputer does not support categorical features and
+possibly creates incorrect values for a categorical feature.</p>
+<p>Note that the mean/median value is computed after filtering out missing values.
+All Null values in the input columns are treated as missing, and so are also imputed. For
+computing median, <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame.approxQuantile" title="pyspark.sql.DataFrame.approxQuantile"><code class="xref py py-meth docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame.approxQuantile()</span></code></a> is used with a
+relative error of <cite>0.001</cite>.</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">df</span> <span class="o">=</span> <span class="n">spark</span><span class="o">.</span><span class="n">createDataFrame</span><span class="p">([(</span><span class="mf">1.0</span><span class="p">,</span> <span class="nb">float</span><span class="p">(</span><span class="s2">"nan"</span><span class="p">)),</span> <span class="p">(</span><span cla [...]
+<span class="gp">... </span> <span class="p">(</span><span class="mf">4.0</span><span class="p">,</span> <span class="mf">4.0</span><span class="p">),</span> <span class="p">(</span><span class="mf">5.0</span><span class="p">,</span> <span class="mf">5.0</span><span class="p">)],</span> <span class="p">[</span><span class="s2">"a"</span><span class="p">,</span> <span class="s2">"b"</span><span class="p">])</span>
+<span class="gp">>>> </span><span class="n">imputer</span> <span class="o">=</span> <span class="n">Imputer</span><span class="p">(</span><span class="n">inputCols</span><span class="o">=</span><span class="p">[</span><span class="s2">"a"</span><span class="p">,</span> <span class="s2">"b"</span><span class="p">],</span> <span class="n">outputCols</span><span class="o">=</span><span class="p">[</span><span class="s2">"out_a"</span><span class="p">,< [...]
+<span class="gp">>>> </span><span class="n">model</span> <span class="o">=</span> <span class="n">imputer</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">df</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">surrogateDF</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
+<span class="go">+---+---+</span>
+<span class="go">| a| b|</span>
+<span class="go">+---+---+</span>
+<span class="go">|3.0|4.0|</span>
+<span class="go">+---+---+</span>
+<span class="gp">...</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
+<span class="go">+---+---+-----+-----+</span>
+<span class="go">| a| b|out_a|out_b|</span>
+<span class="go">+---+---+-----+-----+</span>
+<span class="go">|1.0|NaN| 1.0| 4.0|</span>
+<span class="go">|2.0|NaN| 2.0| 4.0|</span>
+<span class="go">|NaN|3.0| 3.0| 3.0|</span>
+<span class="gp">...</span>
+<span class="gp">>>> </span><span class="n">imputer</span><span class="o">.</span><span class="n">setStrategy</span><span class="p">(</span><span class="s2">"median"</span><span class="p">)</span><span class="o">.</span><span class="n">setMissingValue</span><span class="p">(</span><span class="mf">1.0</span><span class="p">)</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">. [...]
+<span class="go">+---+---+-----+-----+</span>
+<span class="go">| a| b|out_a|out_b|</span>
+<span class="go">+---+---+-----+-----+</span>
+<span class="go">|1.0|NaN| 4.0| NaN|</span>
+<span class="gp">...</span>
+<span class="gp">>>> </span><span class="n">imputerPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/imputer"</span>
+<span class="gp">>>> </span><span class="n">imputer</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">imputerPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedImputer</span> <span class="o">=</span> <span class="n">Imputer</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">imputerPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedImputer</span><span class="o">.</span><span class="n">getStrategy</span><span class="p">()</span> <span class="o">==</span> <span class="n">imputer</span><span class="o">.</span><span class="n">getStrategy</span><span class="p">()</span>
+<span class="go">True</span>
+<span class="gp">>>> </span><span class="n">loadedImputer</span><span class="o">.</span><span class="n">getMissingValue</span><span class="p">()</span>
+<span class="go">1.0</span>
+<span class="gp">>>> </span><span class="n">modelPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/imputer-model"</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">modelPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedModel</span> <span class="o">=</span> <span class="n">ImputerModel</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">modelPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedModel</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">out_a</span> <span class="o">==</span> <span class="n">model</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><spa [...]
+<span class="go">True</span>
+</pre></div>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Imputer.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Imputer.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Imputer.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Imputer.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.fit">
+<code class="descname">fit</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Imputer.fit" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params. If a list/tuple of
+param maps is given, this calls fit on each param map and returns a list of
+models.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>fitted model(s)</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.fitMultiple">
+<code class="descname">fitMultiple</code><span class="sig-paren">(</span><em>dataset</em>, <em>paramMaps</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Imputer.fitMultiple" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset for each param map in <cite>paramMaps</cite>.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a>.</p></li>
+<li><p><strong>paramMaps</strong> – A Sequence of param maps.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>A thread safe iterable which contains one model for each param map. Each
+call to <cite>next(modelIterator)</cite> will return <cite>(index, model)</cite> where model was fit
+using <cite>paramMaps[index]</cite>. <cite>index</cite> values may not be sequential.</p>
+</dd>
+</dl>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>DeveloperApi</p>
+</div>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Experimental</p>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.getInputCols">
+<code class="descname">getInputCols</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Imputer.getInputCols" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of inputCols or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.getMissingValue">
+<code class="descname">getMissingValue</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#Imputer.getMissingValue"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.Imputer.getMissingValue" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of <a class="reference internal" href="#pyspark.ml.feature.Imputer.missingValue" title="pyspark.ml.feature.Imputer.missingValue"><code class="xref py py-attr docutils literal notranslate"><span class="pre">missingValue</span></code></a> or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Imputer.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.getOutputCols">
+<code class="descname">getOutputCols</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#Imputer.getOutputCols"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.Imputer.getOutputCols" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of <a class="reference internal" href="#pyspark.ml.feature.Imputer.outputCols" title="pyspark.ml.feature.Imputer.outputCols"><code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCols</span></code></a> or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Imputer.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.getStrategy">
+<code class="descname">getStrategy</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#Imputer.getStrategy"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.Imputer.getStrategy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of <a class="reference internal" href="#pyspark.ml.feature.Imputer.strategy" title="pyspark.ml.feature.Imputer.strategy"><code class="xref py py-attr docutils literal notranslate"><span class="pre">strategy</span></code></a> or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Imputer.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Imputer.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.Imputer.inputCols">
+<code class="descname">inputCols</code><em class="property"> = Param(parent='undefined', name='inputCols', doc='input column names.')</em><a class="headerlink" href="#pyspark.ml.feature.Imputer.inputCols" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Imputer.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Imputer.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.Imputer.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Imputer.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.Imputer.missingValue">
+<code class="descname">missingValue</code><em class="property"> = Param(parent='undefined', name='missingValue', doc='The placeholder for the missing values. All occurrences of missingValue will be imputed.')</em><a class="headerlink" href="#pyspark.ml.feature.Imputer.missingValue" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.Imputer.outputCols">
+<code class="descname">outputCols</code><em class="property"> = Param(parent='undefined', name='outputCols', doc='output column names.')</em><a class="headerlink" href="#pyspark.ml.feature.Imputer.outputCols" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.Imputer.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.Imputer.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.Imputer.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Imputer.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Imputer.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Imputer.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.setInputCols">
+<code class="descname">setInputCols</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Imputer.setInputCols" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.Imputer.inputCols" title="pyspark.ml.feature.Imputer.inputCols"><code class="xref py py-attr docutils literal notranslate"><span class="pre">inputCols</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.setMissingValue">
+<code class="descname">setMissingValue</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#Imputer.setMissingValue"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.Imputer.setMissingValue" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.Imputer.missingValue" title="pyspark.ml.feature.Imputer.missingValue"><code class="xref py py-attr docutils literal notranslate"><span class="pre">missingValue</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.setOutputCols">
+<code class="descname">setOutputCols</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#Imputer.setOutputCols"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.Imputer.setOutputCols" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.Imputer.outputCols" title="pyspark.ml.feature.Imputer.outputCols"><code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCols</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.setParams">
+<code class="descname">setParams</code><span class="sig-paren">(</span><em>self</em>, <em>strategy="mean"</em>, <em>missingValue=float("nan")</em>, <em>inputCols=None</em>, <em>outputCols=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#Imputer.setParams"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.Imputer.setParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets params for this Imputer.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.setStrategy">
+<code class="descname">setStrategy</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#Imputer.setStrategy"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.Imputer.setStrategy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.Imputer.strategy" title="pyspark.ml.feature.Imputer.strategy"><code class="xref py py-attr docutils literal notranslate"><span class="pre">strategy</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.Imputer.strategy">
+<code class="descname">strategy</code><em class="property"> = Param(parent='undefined', name='strategy', doc='strategy for imputation. If mean, then replace missing values using the mean value of the feature. If median, then replace missing values using the median value of the feature.')</em><a class="headerlink" href="#pyspark.ml.feature.Imputer.strategy" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.Imputer.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.Imputer.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.ImputerModel">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">ImputerModel</code><span class="sig-paren">(</span><em>java_model=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#ImputerModel"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.ImputerModel" title="Permalink to this definition">¶</a></dt>
+<dd><div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Experimental</p>
+</div>
+<p>Model fitted by <a class="reference internal" href="#pyspark.ml.feature.Imputer" title="pyspark.ml.feature.Imputer"><code class="xref py py-class docutils literal notranslate"><span class="pre">Imputer</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.feature.ImputerModel.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ImputerModel.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ImputerModel.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ImputerModel.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ImputerModel.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ImputerModel.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ImputerModel.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ImputerModel.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ImputerModel.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ImputerModel.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ImputerModel.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ImputerModel.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ImputerModel.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ImputerModel.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ImputerModel.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ImputerModel.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ImputerModel.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ImputerModel.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ImputerModel.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ImputerModel.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.ImputerModel.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ImputerModel.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.ImputerModel.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.ImputerModel.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.ImputerModel.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ImputerModel.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ImputerModel.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ImputerModel.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ImputerModel.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ImputerModel.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.ImputerModel.surrogateDF">
+<code class="descname">surrogateDF</code><a class="headerlink" href="#pyspark.ml.feature.ImputerModel.surrogateDF" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns a DataFrame containing inputCols and their corresponding surrogates,
+which are used to replace the missing values in the input DataFrame.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ImputerModel.transform">
+<code class="descname">transform</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ImputerModel.transform" title="Permalink to this definition">¶</a></dt>
+<dd><p>Transforms the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>transformed dataset</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.ImputerModel.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.ImputerModel.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.IndexToString">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">IndexToString</code><span class="sig-paren">(</span><em>inputCol=None</em>, <em>outputCol=None</em>, <em>labels=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#IndexToString"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.IndexToString" title="Permalink to this definition" [...]
+<dd><p>A <code class="xref py py-class docutils literal notranslate"><span class="pre">Transformer</span></code> that maps a column of indices back to a new column of
+corresponding string values.
+The index-string mapping is either from the ML attributes of the input column,
+or from user-supplied labels (which take precedence over ML attributes).
+See <a class="reference internal" href="#pyspark.ml.feature.StringIndexer" title="pyspark.ml.feature.StringIndexer"><code class="xref py py-class docutils literal notranslate"><span class="pre">StringIndexer</span></code></a> for converting strings into indices.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.getInputCol">
+<code class="descname">getInputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.getInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of inputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.getLabels">
+<code class="descname">getLabels</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#IndexToString.getLabels"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.IndexToString.getLabels" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of <a class="reference internal" href="#pyspark.ml.feature.IndexToString.labels" title="pyspark.ml.feature.IndexToString.labels"><code class="xref py py-attr docutils literal notranslate"><span class="pre">labels</span></code></a> or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.getOutputCol">
+<code class="descname">getOutputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.getOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of outputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.IndexToString.inputCol">
+<code class="descname">inputCol</code><em class="property"> = Param(parent='undefined', name='inputCol', doc='input column name.')</em><a class="headerlink" href="#pyspark.ml.feature.IndexToString.inputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.IndexToString.labels">
+<code class="descname">labels</code><em class="property"> = Param(parent='undefined', name='labels', doc='Optional array of labels specifying index-string mapping. If not provided or if empty, then metadata from inputCol is used instead.')</em><a class="headerlink" href="#pyspark.ml.feature.IndexToString.labels" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.IndexToString.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.IndexToString.outputCol">
+<code class="descname">outputCol</code><em class="property"> = Param(parent='undefined', name='outputCol', doc='output column name.')</em><a class="headerlink" href="#pyspark.ml.feature.IndexToString.outputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.IndexToString.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.IndexToString.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.IndexToString.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.setInputCol">
+<code class="descname">setInputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.setInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.IndexToString.inputCol" title="pyspark.ml.feature.IndexToString.inputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">inputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.setLabels">
+<code class="descname">setLabels</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#IndexToString.setLabels"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.IndexToString.setLabels" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.IndexToString.labels" title="pyspark.ml.feature.IndexToString.labels"><code class="xref py py-attr docutils literal notranslate"><span class="pre">labels</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.setOutputCol">
+<code class="descname">setOutputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.setOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.IndexToString.outputCol" title="pyspark.ml.feature.IndexToString.outputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.setParams">
+<code class="descname">setParams</code><span class="sig-paren">(</span><em>self</em>, <em>inputCol=None</em>, <em>outputCol=None</em>, <em>labels=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#IndexToString.setParams"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.IndexToString.setParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets params for this IndexToString.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.transform">
+<code class="descname">transform</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.transform" title="Permalink to this definition">¶</a></dt>
+<dd><p>Transforms the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>transformed dataset</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.IndexToString.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.IndexToString.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.MaxAbsScaler">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">MaxAbsScaler</code><span class="sig-paren">(</span><em>inputCol=None</em>, <em>outputCol=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#MaxAbsScaler"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler" title="Permalink to this definition">¶</a></dt>
+<dd><p>Rescale each feature individually to range [-1, 1] by dividing through the largest maximum
+absolute value in each feature. It does not shift/center the data, and thus does not destroy
+any sparsity.</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">pyspark.ml.linalg</span> <span class="k">import</span> <span class="n">Vectors</span>
+<span class="gp">>>> </span><span class="n">df</span> <span class="o">=</span> <span class="n">spark</span><span class="o">.</span><span class="n">createDataFrame</span><span class="p">([(</span><span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([</span><span class="mf">1.0</span><span class="p">]),),</span> <span class="p">(</span><span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([< [...]
+<span class="gp">>>> </span><span class="n">maScaler</span> <span class="o">=</span> <span class="n">MaxAbsScaler</span><span class="p">(</span><span class="n">inputCol</span><span class="o">=</span><span class="s2">"a"</span><span class="p">,</span> <span class="n">outputCol</span><span class="o">=</span><span class="s2">"scaled"</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">model</span> <span class="o">=</span> <span class="n">maScaler</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">df</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
+<span class="go">+-----+------+</span>
+<span class="go">| a|scaled|</span>
+<span class="go">+-----+------+</span>
+<span class="go">|[1.0]| [0.5]|</span>
+<span class="go">|[2.0]| [1.0]|</span>
+<span class="go">+-----+------+</span>
+<span class="gp">...</span>
+<span class="gp">>>> </span><span class="n">scalerPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/max-abs-scaler"</span>
+<span class="gp">>>> </span><span class="n">maScaler</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">scalerPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedMAScaler</span> <span class="o">=</span> <span class="n">MaxAbsScaler</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">scalerPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedMAScaler</span><span class="o">.</span><span class="n">getInputCol</span><span class="p">()</span> <span class="o">==</span> <span class="n">maScaler</span><span class="o">.</span><span class="n">getInputCol</span><span class="p">()</span>
+<span class="go">True</span>
+<span class="gp">>>> </span><span class="n">loadedMAScaler</span><span class="o">.</span><span class="n">getOutputCol</span><span class="p">()</span> <span class="o">==</span> <span class="n">maScaler</span><span class="o">.</span><span class="n">getOutputCol</span><span class="p">()</span>
+<span class="go">True</span>
+<span class="gp">>>> </span><span class="n">modelPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/max-abs-scaler-model"</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">modelPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedModel</span> <span class="o">=</span> <span class="n">MaxAbsScalerModel</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">modelPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedModel</span><span class="o">.</span><span class="n">maxAbs</span> <span class="o">==</span> <span class="n">model</span><span class="o">.</span><span class="n">maxAbs</span>
+<span class="go">True</span>
+</pre></div>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.fit">
+<code class="descname">fit</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.fit" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params. If a list/tuple of
+param maps is given, this calls fit on each param map and returns a list of
+models.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>fitted model(s)</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.fitMultiple">
+<code class="descname">fitMultiple</code><span class="sig-paren">(</span><em>dataset</em>, <em>paramMaps</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.fitMultiple" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset for each param map in <cite>paramMaps</cite>.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a>.</p></li>
+<li><p><strong>paramMaps</strong> – A Sequence of param maps.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>A thread safe iterable which contains one model for each param map. Each
+call to <cite>next(modelIterator)</cite> will return <cite>(index, model)</cite> where model was fit
+using <cite>paramMaps[index]</cite>. <cite>index</cite> values may not be sequential.</p>
+</dd>
+</dl>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>DeveloperApi</p>
+</div>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Experimental</p>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.getInputCol">
+<code class="descname">getInputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.getInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of inputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.getOutputCol">
+<code class="descname">getOutputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.getOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of outputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.MaxAbsScaler.inputCol">
+<code class="descname">inputCol</code><em class="property"> = Param(parent='undefined', name='inputCol', doc='input column name.')</em><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.inputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.MaxAbsScaler.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.MaxAbsScaler.outputCol">
+<code class="descname">outputCol</code><em class="property"> = Param(parent='undefined', name='outputCol', doc='output column name.')</em><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.outputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.MaxAbsScaler.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.MaxAbsScaler.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.setInputCol">
+<code class="descname">setInputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.setInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.MaxAbsScaler.inputCol" title="pyspark.ml.feature.MaxAbsScaler.inputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">inputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.setOutputCol">
+<code class="descname">setOutputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.setOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.MaxAbsScaler.outputCol" title="pyspark.ml.feature.MaxAbsScaler.outputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.setParams">
+<code class="descname">setParams</code><span class="sig-paren">(</span><em>self</em>, <em>inputCol=None</em>, <em>outputCol=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#MaxAbsScaler.setParams"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.setParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets params for this MaxAbsScaler.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScaler.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScaler.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.MaxAbsScalerModel">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">MaxAbsScalerModel</code><span class="sig-paren">(</span><em>java_model=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#MaxAbsScalerModel"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScalerModel" title="Permalink to this definition">¶</a></dt>
+<dd><p>Model fitted by <a class="reference internal" href="#pyspark.ml.feature.MaxAbsScaler" title="pyspark.ml.feature.MaxAbsScaler"><code class="xref py py-class docutils literal notranslate"><span class="pre">MaxAbsScaler</span></code></a>.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScalerModel.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScalerModel.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScalerModel.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScalerModel.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScalerModel.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScalerModel.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScalerModel.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScalerModel.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScalerModel.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScalerModel.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScalerModel.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScalerModel.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScalerModel.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScalerModel.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScalerModel.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScalerModel.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScalerModel.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScalerModel.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScalerModel.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScalerModel.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.MaxAbsScalerModel.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScalerModel.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.MaxAbsScalerModel.maxAbs">
+<code class="descname">maxAbs</code><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScalerModel.maxAbs" title="Permalink to this definition">¶</a></dt>
+<dd><p>Max Abs vector.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.0.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.MaxAbsScalerModel.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScalerModel.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.MaxAbsScalerModel.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScalerModel.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScalerModel.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScalerModel.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScalerModel.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScalerModel.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScalerModel.transform">
+<code class="descname">transform</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScalerModel.transform" title="Permalink to this definition">¶</a></dt>
+<dd><p>Transforms the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>transformed dataset</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MaxAbsScalerModel.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MaxAbsScalerModel.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.MinHashLSH">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">MinHashLSH</code><span class="sig-paren">(</span><em>inputCol=None</em>, <em>outputCol=None</em>, <em>seed=None</em>, <em>numHashTables=1</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#MinHashLSH"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH" title="Permalink to t [...]
+<dd><div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Experimental</p>
+</div>
+<p>LSH class for Jaccard distance.
+The input can be dense or sparse vectors, but it is more efficient if it is sparse.
+For example, <cite>Vectors.sparse(10, [(2, 1.0), (3, 1.0), (5, 1.0)])</cite> means there are 10 elements
+in the space. This set contains elements 2, 3, and 5. Also, any input vector must have at
+least 1 non-zero index, and all non-zero values are treated as binary “1” values.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
+<p><a class="reference external" href="https://en.wikipedia.org/wiki/MinHash">Wikipedia on MinHash</a></p>
+</div>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">pyspark.ml.linalg</span> <span class="k">import</span> <span class="n">Vectors</span>
+<span class="gp">>>> </span><span class="kn">from</span> <span class="nn">pyspark.sql.functions</span> <span class="k">import</span> <span class="n">col</span>
+<span class="gp">>>> </span><span class="n">data</span> <span class="o">=</span> <span class="p">[(</span><span class="mi">0</span><span class="p">,</span> <span class="n">Vectors</span><span class="o">.</span><span class="n">sparse</span><span class="p">(</span><span class="mi">6</span><span class="p">,</span> <span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">],</s [...]
+<span class="gp">... </span> <span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">Vectors</span><span class="o">.</span><span class="n">sparse</span><span class="p">(</span><span class="mi">6</span><span class="p">,</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">],</span> <span class="p">[</span><span class="mf">1.0</span [...]
+<span class="gp">... </span> <span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="n">Vectors</span><span class="o">.</span><span class="n">sparse</span><span class="p">(</span><span class="mi">6</span><span class="p">,</span> <span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">],</span> <span class="p">[</span><span class="mf">1.0</span [...]
+<span class="gp">>>> </span><span class="n">df</span> <span class="o">=</span> <span class="n">spark</span><span class="o">.</span><span class="n">createDataFrame</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="p">[</span><span class="s2">"id"</span><span class="p">,</span> <span class="s2">"features"</span><span class="p">])</span>
+<span class="gp">>>> </span><span class="n">mh</span> <span class="o">=</span> <span class="n">MinHashLSH</span><span class="p">(</span><span class="n">inputCol</span><span class="o">=</span><span class="s2">"features"</span><span class="p">,</span> <span class="n">outputCol</span><span class="o">=</span><span class="s2">"hashes"</span><span class="p">,</span> <span class="n">seed</span><span class="o">=</span><span class="mi">12345</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">model</span> <span class="o">=</span> <span class="n">mh</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">df</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span>
+<span class="go">Row(id=0, features=SparseVector(6, {0: 1.0, 1: 1.0, 2: 1.0}), hashes=[DenseVector([6179668...</span>
+<span class="gp">>>> </span><span class="n">data2</span> <span class="o">=</span> <span class="p">[(</span><span class="mi">3</span><span class="p">,</span> <span class="n">Vectors</span><span class="o">.</span><span class="n">sparse</span><span class="p">(</span><span class="mi">6</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">],</ [...]
+<span class="gp">... </span> <span class="p">(</span><span class="mi">4</span><span class="p">,</span> <span class="n">Vectors</span><span class="o">.</span><span class="n">sparse</span><span class="p">(</span><span class="mi">6</span><span class="p">,</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">],</span> <span class="p">[</span><span class="mf">1.0</spa [...]
+<span class="gp">... </span> <span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="n">Vectors</span><span class="o">.</span><span class="n">sparse</span><span class="p">(</span><span class="mi">6</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">],</span> <span class="p">[</span><span class="mf">1.0</spa [...]
+<span class="gp">>>> </span><span class="n">df2</span> <span class="o">=</span> <span class="n">spark</span><span class="o">.</span><span class="n">createDataFrame</span><span class="p">(</span><span class="n">data2</span><span class="p">,</span> <span class="p">[</span><span class="s2">"id"</span><span class="p">,</span> <span class="s2">"features"</span><span class="p">])</span>
+<span class="gp">>>> </span><span class="n">key</span> <span class="o">=</span> <span class="n">Vectors</span><span class="o">.</span><span class="n">sparse</span><span class="p">(</span><span class="mi">6</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">],</span> <span class="p">[</span><span class="mf">1.0</span><span class="p">,</span> <span class="mf">1.0</span><span class="p">])</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">approxNearestNeighbors</span><span class="p">(</span><span class="n">df2</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">collect</span><span class="p">()</span>
+<span class="go">[Row(id=5, features=SparseVector(6, {1: 1.0, 2: 1.0, 4: 1.0}), hashes=[DenseVector([6179668...</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">approxSimilarityJoin</span><span class="p">(</span><span class="n">df</span><span class="p">,</span> <span class="n">df2</span><span class="p">,</span> <span class="mf">0.6</span><span class="p">,</span> <span class="n">distCol</span><span class="o">=</span><span class="s2">"JaccardDistance"</span><span class="p">)</span><span class="o">.</span><span class="n">select</span>< [...]
+<span class="gp">... </span> <span class="n">col</span><span class="p">(</span><span class="s2">"datasetA.id"</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">"idA"</span><span class="p">),</span>
+<span class="gp">... </span> <span class="n">col</span><span class="p">(</span><span class="s2">"datasetB.id"</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">"idB"</span><span class="p">),</span>
+<span class="gp">... </span> <span class="n">col</span><span class="p">(</span><span class="s2">"JaccardDistance"</span><span class="p">))</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
+<span class="go">+---+---+---------------+</span>
+<span class="go">|idA|idB|JaccardDistance|</span>
+<span class="go">+---+---+---------------+</span>
+<span class="go">| 0| 5| 0.5|</span>
+<span class="go">| 1| 4| 0.5|</span>
+<span class="go">+---+---+---------------+</span>
+<span class="gp">...</span>
+<span class="gp">>>> </span><span class="n">mhPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/mh"</span>
+<span class="gp">>>> </span><span class="n">mh</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">mhPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">mh2</span> <span class="o">=</span> <span class="n">MinHashLSH</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">mhPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">mh2</span><span class="o">.</span><span class="n">getOutputCol</span><span class="p">()</span> <span class="o">==</span> <span class="n">mh</span><span class="o">.</span><span class="n">getOutputCol</span><span class="p">()</span>
+<span class="go">True</span>
+<span class="gp">>>> </span><span class="n">modelPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/mh-model"</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">modelPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">model2</span> <span class="o">=</span> <span class="n">MinHashLSHModel</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">modelPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">hashes</span> <span class="o">==</span> <span class="n">model2</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span cl [...]
+<span class="go">True</span>
+</pre></div>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.fit">
+<code class="descname">fit</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.fit" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params. If a list/tuple of
+param maps is given, this calls fit on each param map and returns a list of
+models.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>fitted model(s)</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.fitMultiple">
+<code class="descname">fitMultiple</code><span class="sig-paren">(</span><em>dataset</em>, <em>paramMaps</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.fitMultiple" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset for each param map in <cite>paramMaps</cite>.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a>.</p></li>
+<li><p><strong>paramMaps</strong> – A Sequence of param maps.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>A thread safe iterable which contains one model for each param map. Each
+call to <cite>next(modelIterator)</cite> will return <cite>(index, model)</cite> where model was fit
+using <cite>paramMaps[index]</cite>. <cite>index</cite> values may not be sequential.</p>
+</dd>
+</dl>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>DeveloperApi</p>
+</div>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Experimental</p>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.getInputCol">
+<code class="descname">getInputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.getInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of inputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.getNumHashTables">
+<code class="descname">getNumHashTables</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.getNumHashTables" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of numHashTables or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.getOutputCol">
+<code class="descname">getOutputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.getOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of outputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.getSeed">
+<code class="descname">getSeed</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.getSeed" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of seed or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.MinHashLSH.inputCol">
+<code class="descname">inputCol</code><em class="property"> = Param(parent='undefined', name='inputCol', doc='input column name.')</em><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.inputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.MinHashLSH.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.MinHashLSH.numHashTables">
+<code class="descname">numHashTables</code><em class="property"> = Param(parent='undefined', name='numHashTables', doc='number of hash tables, where increasing number of hash tables lowers the false negative rate, and decreasing it improves the running performance.')</em><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.numHashTables" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.MinHashLSH.outputCol">
+<code class="descname">outputCol</code><em class="property"> = Param(parent='undefined', name='outputCol', doc='output column name.')</em><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.outputCol" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.MinHashLSH.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.MinHashLSH.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.MinHashLSH.seed">
+<code class="descname">seed</code><em class="property"> = Param(parent='undefined', name='seed', doc='random seed.')</em><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.seed" title="Permalink to this definition">¶</a></dt>
+<dd></dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.setInputCol">
+<code class="descname">setInputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.setInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.MinHashLSH.inputCol" title="pyspark.ml.feature.MinHashLSH.inputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">inputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.setNumHashTables">
+<code class="descname">setNumHashTables</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.setNumHashTables" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.MinHashLSH.numHashTables" title="pyspark.ml.feature.MinHashLSH.numHashTables"><code class="xref py py-attr docutils literal notranslate"><span class="pre">numHashTables</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.setOutputCol">
+<code class="descname">setOutputCol</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.setOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.MinHashLSH.outputCol" title="pyspark.ml.feature.MinHashLSH.outputCol"><code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.setParams">
+<code class="descname">setParams</code><span class="sig-paren">(</span><em>self</em>, <em>inputCol=None</em>, <em>outputCol=None</em>, <em>seed=None</em>, <em>numHashTables=1</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#MinHashLSH.setParams"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.setParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets params for this MinHashLSH.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.setSeed">
+<code class="descname">setSeed</code><span class="sig-paren">(</span><em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.setSeed" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.MinHashLSH.seed" title="pyspark.ml.feature.MinHashLSH.seed"><code class="xref py py-attr docutils literal notranslate"><span class="pre">seed</span></code></a>.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSH.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSH.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.MinHashLSHModel">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">MinHashLSHModel</code><span class="sig-paren">(</span><em>java_model=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#MinHashLSHModel"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel" title="Permalink to this definition">¶</a></dt>
+<dd><div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Experimental</p>
+</div>
+<p>Model produced by <a class="reference internal" href="#pyspark.ml.feature.MinHashLSH" title="pyspark.ml.feature.MinHashLSH"><code class="xref py py-class docutils literal notranslate"><span class="pre">MinHashLSH</span></code></a>, where where multiple hash functions are stored. Each
+hash function is picked from the following family of hash functions, where <span class="math notranslate nohighlight">\(a_i\)</span> and
+<span class="math notranslate nohighlight">\(b_i\)</span> are randomly chosen integers less than prime:
+<span class="math notranslate nohighlight">\(h_i(x) = ((x \cdot a_i + b_i) \mod prime)\)</span> This hash family is approximately min-wise
+independent according to the reference.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
+<p>Tom Bohman, Colin Cooper, and Alan Frieze. “Min-wise independent linear
+permutations.” Electronic Journal of Combinatorics 7 (2000): R26.</p>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.2.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSHModel.approxNearestNeighbors">
+<code class="descname">approxNearestNeighbors</code><span class="sig-paren">(</span><em>dataset</em>, <em>key</em>, <em>numNearestNeighbors</em>, <em>distCol='distCol'</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel.approxNearestNeighbors" title="Permalink to this definition">¶</a></dt>
+<dd><p>Given a large dataset and an item, approximately find at most k items which have the
+closest distance to the item. If the <code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code> is missing, the method will
+transform the data; if the <code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code> exists, it will use that. This allows
+caching of the transformed data when necessary.</p>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>This method is experimental and will likely change behavior in the next release.</p>
+</div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – The dataset to search for nearest neighbors of the key.</p></li>
+<li><p><strong>key</strong> – Feature vector representing the item to search for.</p></li>
+<li><p><strong>numNearestNeighbors</strong> – The maximum number of nearest neighbors.</p></li>
+<li><p><strong>distCol</strong> – Output column for storing the distance between each result row and the key.
+Use “distCol” as default value if it’s not specified.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>A dataset containing at most k items closest to the key. A column “distCol” is
+added to show the distance between each row and the key.</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSHModel.approxSimilarityJoin">
+<code class="descname">approxSimilarityJoin</code><span class="sig-paren">(</span><em>datasetA</em>, <em>datasetB</em>, <em>threshold</em>, <em>distCol='distCol'</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel.approxSimilarityJoin" title="Permalink to this definition">¶</a></dt>
+<dd><p>Join two datasets to approximately find all pairs of rows whose distance are smaller than
+the threshold. If the <code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code> is missing, the method will transform the data;
+if the <code class="xref py py-attr docutils literal notranslate"><span class="pre">outputCol</span></code> exists, it will use that. This allows caching of the
+transformed data when necessary.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>datasetA</strong> – One of the datasets to join.</p></li>
+<li><p><strong>datasetB</strong> – Another dataset to join.</p></li>
+<li><p><strong>threshold</strong> – The threshold for the distance of row pairs.</p></li>
+<li><p><strong>distCol</strong> – Output column for storing the distance between each pair of rows. Use
+“distCol” as default value if it’s not specified.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>A joined dataset containing pairs of rows. The original rows are in columns
+“datasetA” and “datasetB”, and a column “distCol” is added to show the distance
+between each pair.</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSHModel.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSHModel.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSHModel.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSHModel.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSHModel.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSHModel.getParam">
+<code class="descname">getParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel.getParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets a param by its name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSHModel.hasDefault">
+<code class="descname">hasDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel.hasDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param has a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSHModel.hasParam">
+<code class="descname">hasParam</code><span class="sig-paren">(</span><em>paramName</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel.hasParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Tests whether this instance contains a param with a given
+(string) name.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSHModel.isDefined">
+<code class="descname">isDefined</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel.isDefined" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user or has
+a default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSHModel.isSet">
+<code class="descname">isSet</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel.isSet" title="Permalink to this definition">¶</a></dt>
+<dd><p>Checks whether a param is explicitly set by user.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.MinHashLSHModel.load">
+<em class="property">classmethod </em><code class="descname">load</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel.load" title="Permalink to this definition">¶</a></dt>
+<dd><p>Reads an ML instance from the input path, a shortcut of <cite>read().load(path)</cite>.</p>
+</dd></dl>
+
+<dl class="attribute">
+<dt id="pyspark.ml.feature.MinHashLSHModel.params">
+<code class="descname">params</code><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel.params" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns all params ordered by name. The default implementation
+uses <code class="xref py py-func docutils literal notranslate"><span class="pre">dir()</span></code> to get all attributes of type
+<code class="xref py py-class docutils literal notranslate"><span class="pre">Param</span></code>.</p>
+</dd></dl>
+
+<dl class="classmethod">
+<dt id="pyspark.ml.feature.MinHashLSHModel.read">
+<em class="property">classmethod </em><code class="descname">read</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel.read" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLReader instance for this class.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSHModel.save">
+<code class="descname">save</code><span class="sig-paren">(</span><em>path</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel.save" title="Permalink to this definition">¶</a></dt>
+<dd><p>Save this ML instance to the given path, a shortcut of ‘write().save(path)’.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSHModel.set">
+<code class="descname">set</code><span class="sig-paren">(</span><em>param</em>, <em>value</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel.set" title="Permalink to this definition">¶</a></dt>
+<dd><p>Sets a parameter in the embedded param map.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSHModel.transform">
+<code class="descname">transform</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel.transform" title="Permalink to this definition">¶</a></dt>
+<dd><p>Transforms the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>transformed dataset</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinHashLSHModel.write">
+<code class="descname">write</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinHashLSHModel.write" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns an MLWriter instance for this ML instance.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="class">
+<dt id="pyspark.ml.feature.MinMaxScaler">
+<em class="property">class </em><code class="descclassname">pyspark.ml.feature.</code><code class="descname">MinMaxScaler</code><span class="sig-paren">(</span><em>min=0.0</em>, <em>max=1.0</em>, <em>inputCol=None</em>, <em>outputCol=None</em><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#MinMaxScaler"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.MinMaxScaler" title="Permalink to this [...]
+<dd><p>Rescale each feature individually to a common range [min, max] linearly using column summary
+statistics, which is also known as min-max normalization or Rescaling. The rescaled value for
+feature E is calculated as,</p>
+<p>Rescaled(e_i) = (e_i - E_min) / (E_max - E_min) * (max - min) + min</p>
+<p>For the case E_max == E_min, Rescaled(e_i) = 0.5 * (max + min)</p>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Since zero values will probably be transformed to non-zero values, output of the
+transformer will be DenseVector even for sparse input.</p>
+</div>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">pyspark.ml.linalg</span> <span class="k">import</span> <span class="n">Vectors</span>
+<span class="gp">>>> </span><span class="n">df</span> <span class="o">=</span> <span class="n">spark</span><span class="o">.</span><span class="n">createDataFrame</span><span class="p">([(</span><span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([</span><span class="mf">0.0</span><span class="p">]),),</span> <span class="p">(</span><span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">([< [...]
+<span class="gp">>>> </span><span class="n">mmScaler</span> <span class="o">=</span> <span class="n">MinMaxScaler</span><span class="p">(</span><span class="n">inputCol</span><span class="o">=</span><span class="s2">"a"</span><span class="p">,</span> <span class="n">outputCol</span><span class="o">=</span><span class="s2">"scaled"</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">model</span> <span class="o">=</span> <span class="n">mmScaler</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">df</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">originalMin</span>
+<span class="go">DenseVector([0.0])</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">originalMax</span>
+<span class="go">DenseVector([2.0])</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
+<span class="go">+-----+------+</span>
+<span class="go">| a|scaled|</span>
+<span class="go">+-----+------+</span>
+<span class="go">|[0.0]| [0.0]|</span>
+<span class="go">|[2.0]| [1.0]|</span>
+<span class="go">+-----+------+</span>
+<span class="gp">...</span>
+<span class="gp">>>> </span><span class="n">minMaxScalerPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/min-max-scaler"</span>
+<span class="gp">>>> </span><span class="n">mmScaler</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">minMaxScalerPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedMMScaler</span> <span class="o">=</span> <span class="n">MinMaxScaler</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">minMaxScalerPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedMMScaler</span><span class="o">.</span><span class="n">getMin</span><span class="p">()</span> <span class="o">==</span> <span class="n">mmScaler</span><span class="o">.</span><span class="n">getMin</span><span class="p">()</span>
+<span class="go">True</span>
+<span class="gp">>>> </span><span class="n">loadedMMScaler</span><span class="o">.</span><span class="n">getMax</span><span class="p">()</span> <span class="o">==</span> <span class="n">mmScaler</span><span class="o">.</span><span class="n">getMax</span><span class="p">()</span>
+<span class="go">True</span>
+<span class="gp">>>> </span><span class="n">modelPath</span> <span class="o">=</span> <span class="n">temp_path</span> <span class="o">+</span> <span class="s2">"/min-max-scaler-model"</span>
+<span class="gp">>>> </span><span class="n">model</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">modelPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedModel</span> <span class="o">=</span> <span class="n">MinMaxScalerModel</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">modelPath</span><span class="p">)</span>
+<span class="gp">>>> </span><span class="n">loadedModel</span><span class="o">.</span><span class="n">originalMin</span> <span class="o">==</span> <span class="n">model</span><span class="o">.</span><span class="n">originalMin</span>
+<span class="go">True</span>
+<span class="gp">>>> </span><span class="n">loadedModel</span><span class="o">.</span><span class="n">originalMax</span> <span class="o">==</span> <span class="n">model</span><span class="o">.</span><span class="n">originalMax</span>
+<span class="go">True</span>
+</pre></div>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+<dl class="method">
+<dt id="pyspark.ml.feature.MinMaxScaler.copy">
+<code class="descname">copy</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinMaxScaler.copy" title="Permalink to this definition">¶</a></dt>
+<dd><p>Creates a copy of this instance with the same uid and some
+extra params. This implementation first calls Params.copy and
+then make a copy of the companion Java pipeline component with
+extra params. So both the Python wrapper and the Java pipeline
+component get copied.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – Extra parameters to copy to the new instance</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Copy of this instance</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinMaxScaler.explainParam">
+<code class="descname">explainParam</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinMaxScaler.explainParam" title="Permalink to this definition">¶</a></dt>
+<dd><p>Explains a single param and returns its name, doc, and optional
+default value and user-supplied value in a string.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinMaxScaler.explainParams">
+<code class="descname">explainParams</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinMaxScaler.explainParams" title="Permalink to this definition">¶</a></dt>
+<dd><p>Returns the documentation of all params with their optionally
+default values and user-supplied values.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinMaxScaler.extractParamMap">
+<code class="descname">extractParamMap</code><span class="sig-paren">(</span><em>extra=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinMaxScaler.extractParamMap" title="Permalink to this definition">¶</a></dt>
+<dd><p>Extracts the embedded default param values and user-supplied
+values, and then merges them with extra values from input into
+a flat param map, where the latter value is used if there exist
+conflicts, i.e., with ordering: default param values <
+user-supplied values < extra.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>extra</strong> – extra param values</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>merged param map</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinMaxScaler.fit">
+<code class="descname">fit</code><span class="sig-paren">(</span><em>dataset</em>, <em>params=None</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinMaxScaler.fit" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset with optional parameters.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a></p></li>
+<li><p><strong>params</strong> – an optional param map that overrides embedded params. If a list/tuple of
+param maps is given, this calls fit on each param map and returns a list of
+models.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>fitted model(s)</p>
+</dd>
+</dl>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinMaxScaler.fitMultiple">
+<code class="descname">fitMultiple</code><span class="sig-paren">(</span><em>dataset</em>, <em>paramMaps</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinMaxScaler.fitMultiple" title="Permalink to this definition">¶</a></dt>
+<dd><p>Fits a model to the input dataset for each param map in <cite>paramMaps</cite>.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>dataset</strong> – input dataset, which is an instance of <a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">pyspark.sql.DataFrame</span></code></a>.</p></li>
+<li><p><strong>paramMaps</strong> – A Sequence of param maps.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>A thread safe iterable which contains one model for each param map. Each
+call to <cite>next(modelIterator)</cite> will return <cite>(index, model)</cite> where model was fit
+using <cite>paramMaps[index]</cite>. <cite>index</cite> values may not be sequential.</p>
+</dd>
+</dl>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>DeveloperApi</p>
+</div>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Experimental</p>
+</div>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 2.3.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinMaxScaler.getInputCol">
+<code class="descname">getInputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinMaxScaler.getInputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of inputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinMaxScaler.getMax">
+<code class="descname">getMax</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#MinMaxScaler.getMax"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.MinMaxScaler.getMax" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of max or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinMaxScaler.getMin">
+<code class="descname">getMin</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="_modules/pyspark/ml/feature.html#MinMaxScaler.getMin"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.MinMaxScaler.getMin" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of min or its default value.</p>
+<div class="versionadded">
+<p><span class="versionmodified added">New in version 1.6.0.</span></p>
+</div>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinMaxScaler.getOrDefault">
+<code class="descname">getOrDefault</code><span class="sig-paren">(</span><em>param</em><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinMaxScaler.getOrDefault" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of a param in the user-supplied param map or its
+default value. Raises an error if neither is set.</p>
+</dd></dl>
+
+<dl class="method">
+<dt id="pyspark.ml.feature.MinMaxScaler.getOutputCol">
+<code class="descname">getOutputCol</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pyspark.ml.feature.MinMaxScaler.getOutputCol" title="Permalink to this definition">¶</a></dt>
+<dd><p>Gets the value of outputCol or its default value.</p>
+</dd></dl>
+
+<dl class="method">
... 37089 lines suppressed ...
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org