You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sr...@apache.org on 2016/12/21 08:59:43 UTC
spark git commit: [SPARK-18923][DOC][BUILD] Support skipping R/Python
API docs
Repository: spark
Updated Branches:
refs/heads/master 24c0c9412 -> ba4468bb2
[SPARK-18923][DOC][BUILD] Support skipping R/Python API docs
## What changes were proposed in this pull request?
We can build Python API docs by `cd ./python/docs && make html for Python` and R API docs by `cd ./R && sh create-docs.sh for R` separately. However, `jekyll` fails in some environments.
This PR aims to support `SKIP_PYTHONDOC` and `SKIP_RDOC` for documentation build in `docs` folder. Currently, we can use `SKIP_SCALADOC` or `SKIP_API`. The reason providing additional options is that the Spark documentation build uses a number of tools to build HTML docs and API docs in Scala, Python and R. Specifically, for Python and R,
- Python API docs requires `sphinx`.
- R API docs requires `R` installation and `knitr` (and more others libraries).
In other words, we cannot generate Python API docs without R installation. Also, we cannot generate R API docs without Python `sphinx` installation. If Spark provides `SKIP_PYTHONDOC` and `SKIP_RDOC` like `SKIP_SCALADOC`, it would be more convenient.
## How was this patch tested?
Manual.
**Skipping Scala/Java/Python API Doc Build**
```bash
$ cd docs
$ SKIP_SCALADOC=1 SKIP_PYTHONDOC=1 jekyll build
$ ls api
DESCRIPTION R
```
**Skipping Scala/Java/R API Doc Build**
```bash
$ cd docs
$ SKIP_SCALADOC=1 SKIP_RDOC=1 jekyll build
$ ls api
python
```
Author: Dongjoon Hyun <do...@apache.org>
Closes #16336 from dongjoon-hyun/SPARK-18923.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/ba4468bb
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/ba4468bb
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/ba4468bb
Branch: refs/heads/master
Commit: ba4468bb24f2e13b98e7b66b44203c20393b8ab8
Parents: 24c0c94
Author: Dongjoon Hyun <do...@apache.org>
Authored: Wed Dec 21 08:59:38 2016 +0000
Committer: Sean Owen <so...@cloudera.com>
Committed: Wed Dec 21 08:59:38 2016 +0000
----------------------------------------------------------------------
docs/README.md | 3 ++-
docs/_plugins/copy_api_dirs.rb | 49 ++++++++++++++++++++-----------------
2 files changed, 29 insertions(+), 23 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/ba4468bb/docs/README.md
----------------------------------------------------------------------
diff --git a/docs/README.md b/docs/README.md
index ffd3b57..90e10a1 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -69,4 +69,5 @@ may take some time as it generates all of the scaladoc. The jekyll plugin also
PySpark docs using [Sphinx](http://sphinx-doc.org/).
NOTE: To skip the step of building and copying over the Scala, Python, R API docs, run `SKIP_API=1
-jekyll`.
+jekyll`. In addition, `SKIP_SCALADOC=1`, `SKIP_PYTHONDOC=1`, and `SKIP_RDOC=1` can be used to skip a single
+step of the corresponding language.
http://git-wip-us.apache.org/repos/asf/spark/blob/ba4468bb/docs/_plugins/copy_api_dirs.rb
----------------------------------------------------------------------
diff --git a/docs/_plugins/copy_api_dirs.rb b/docs/_plugins/copy_api_dirs.rb
index 71e6432..95e3ba3 100644
--- a/docs/_plugins/copy_api_dirs.rb
+++ b/docs/_plugins/copy_api_dirs.rb
@@ -113,36 +113,41 @@ if not (ENV['SKIP_API'] == '1')
File.open(css_file, 'a') { |f| f.write("\n" + css.join()) }
end
- # Build Sphinx docs for Python
+ if not (ENV['SKIP_PYTHONDOC'] == '1')
+ # Build Sphinx docs for Python
- puts "Moving to python/docs directory and building sphinx."
- cd("../python/docs")
- system("make html") || raise("Python doc generation failed")
+ puts "Moving to python/docs directory and building sphinx."
+ cd("../python/docs")
+ system("make html") || raise("Python doc generation failed")
- puts "Moving back into home dir."
- cd("../../")
+ puts "Moving back into docs dir."
+ cd("../../docs")
- puts "Making directory api/python"
- mkdir_p "docs/api/python"
+ puts "Making directory api/python"
+ mkdir_p "api/python"
- puts "cp -r python/docs/_build/html/. docs/api/python"
- cp_r("python/docs/_build/html/.", "docs/api/python")
+ puts "cp -r ../python/docs/_build/html/. api/python"
+ cp_r("../python/docs/_build/html/.", "api/python")
+ end
- # Build SparkR API docs
- puts "Moving to R directory and building roxygen docs."
- cd("R")
- system("./create-docs.sh") || raise("R doc generation failed")
+ if not (ENV['SKIP_RDOC'] == '1')
+ # Build SparkR API docs
- puts "Moving back into home dir."
- cd("../")
+ puts "Moving to R directory and building roxygen docs."
+ cd("../R")
+ system("./create-docs.sh") || raise("R doc generation failed")
- puts "Making directory api/R"
- mkdir_p "docs/api/R"
+ puts "Moving back into docs dir."
+ cd("../docs")
- puts "cp -r R/pkg/html/. docs/api/R"
- cp_r("R/pkg/html/.", "docs/api/R")
+ puts "Making directory api/R"
+ mkdir_p "api/R"
- puts "cp R/pkg/DESCRIPTION docs/api"
- cp("R/pkg/DESCRIPTION", "docs/api")
+ puts "cp -r ../R/pkg/html/. api/R"
+ cp_r("../R/pkg/html/.", "api/R")
+
+ puts "cp ../R/pkg/DESCRIPTION api"
+ cp("../R/pkg/DESCRIPTION", "api")
+ end
end
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org