You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@bahir.apache.org by lr...@apache.org on 2016/08/10 21:13:50 UTC
bahir-website git commit: [BAHIR-50] Publish extensions documentation
to website
Repository: bahir-website
Updated Branches:
refs/heads/master d09e63b3b -> f8d356989
[BAHIR-50] Publish extensions documentation to website
Provide a script to gather extension documentation based
on its README.md and publish it as part of the current
documentation on the website.
When a release is performed, the "current" folder should
be copied to the release name (e.g. 2.0.0) as a documentation
snapshot.
Project: http://git-wip-us.apache.org/repos/asf/bahir-website/repo
Commit: http://git-wip-us.apache.org/repos/asf/bahir-website/commit/f8d35698
Tree: http://git-wip-us.apache.org/repos/asf/bahir-website/tree/f8d35698
Diff: http://git-wip-us.apache.org/repos/asf/bahir-website/diff/f8d35698
Branch: refs/heads/master
Commit: f8d356989c150427f4d85c1ff12e2f06864f4d63
Parents: d09e63b
Author: Luciano Resende <lr...@apache.org>
Authored: Sun Aug 7 19:53:11 2016 +0300
Committer: Luciano Resende <lr...@apache.org>
Committed: Wed Aug 10 14:13:21 2016 -0700
----------------------------------------------------------------------
README.md | 21 ++++++-
site/_data/navigation.yml | 5 ++
site/docs/spark/current/.gitignore | 1 +
site/docs/spark/current/documentation.md | 46 +++++++++++++++
.../current/spark-sql-streaming-mqtt.template | 27 +++++++++
.../spark/current/spark-streaming-akka.template | 26 +++++++++
.../spark/current/spark-streaming-mqtt.template | 27 +++++++++
.../current/spark-streaming-twitter.template | 26 +++++++++
.../current/spark-streaming-zeromq.template | 26 +++++++++
update-doc.sh | 61 ++++++++++++++++++++
10 files changed, 264 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/bahir-website/blob/f8d35698/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 418049f..c72acee 100644
--- a/README.md
+++ b/README.md
@@ -37,6 +37,22 @@ running from within the directory:
```
+## Update extensions documentation
+
+To update the current documentation contents, use the update-doc script.
+This script will update the documentation contents based on the README.md
+files from the source of each extension.
+
+```
+1. update-doc.sh
+2. git commit -a -m"My updates"
+3. git push
+4. ./publish.sh
+5. git push origin asf-site
+```
+
+
+
## Publishing to live site
Bahir is using [gitpubsub](http://www.apache.org/dev/gitpubsub.html) for publishing the website,
@@ -48,8 +64,9 @@ shell script.
```
1. Make your changes
2. git commit -a -m"My updates"
-3. ./publish.sh
-4. git push origin asf-site
+3. git push
+4. ./publish.sh
+5. git push origin asf-site
```
Within a few minutes, gitpubsub should kick in and you'll be able to
http://git-wip-us.apache.org/repos/asf/bahir-website/blob/f8d35698/site/_data/navigation.yml
----------------------------------------------------------------------
diff --git a/site/_data/navigation.yml b/site/_data/navigation.yml
index 8aa4588..16ccd13 100644
--- a/site/_data/navigation.yml
+++ b/site/_data/navigation.yml
@@ -45,6 +45,11 @@ topnav:
- title: Project Committers
url: /community-members
+- title: Documentation
+ subcategories:
+ - title: Current
+ url: /docs/spark/current/documentation
+
- title: GitHub
url: https://github.com/apache/bahir
http://git-wip-us.apache.org/repos/asf/bahir-website/blob/f8d35698/site/docs/spark/current/.gitignore
----------------------------------------------------------------------
diff --git a/site/docs/spark/current/.gitignore b/site/docs/spark/current/.gitignore
new file mode 100644
index 0000000..6b7415a
--- /dev/null
+++ b/site/docs/spark/current/.gitignore
@@ -0,0 +1 @@
+spark*.md
http://git-wip-us.apache.org/repos/asf/bahir-website/blob/f8d35698/site/docs/spark/current/documentation.md
----------------------------------------------------------------------
diff --git a/site/docs/spark/current/documentation.md b/site/docs/spark/current/documentation.md
new file mode 100644
index 0000000..16b6527
--- /dev/null
+++ b/site/docs/spark/current/documentation.md
@@ -0,0 +1,46 @@
+---
+layout: page
+title: Extensions for Apache Spark
+description: Extensions for Apache Spark
+group: nav-right
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+{% include JB/setup %}
+
+### Apache Bahir Extensions for Apache Spark
+
+<br/>
+
+#### Structured Streaming Data Sources
+
+[MQTT data source](/docs/spark/current/spark-sql-streaming-mqtt)
+
+<br/>
+
+#### Discretized Streams (DStreams) Connectors
+
+[Akka connector](/docs/spark/current/spark-streaming-akka)
+
+[MQTT connector](/docs/spark/current/spark-streaming-mqtt)
+
+[Twitter connector](/docs/spark/current/spark-streaming-twitter)
+
+[ZeroMQ connector](/docs/spark/current/spark-streaming-zeromq)
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/bahir-website/blob/f8d35698/site/docs/spark/current/spark-sql-streaming-mqtt.template
----------------------------------------------------------------------
diff --git a/site/docs/spark/current/spark-sql-streaming-mqtt.template b/site/docs/spark/current/spark-sql-streaming-mqtt.template
new file mode 100644
index 0000000..fd08baa
--- /dev/null
+++ b/site/docs/spark/current/spark-sql-streaming-mqtt.template
@@ -0,0 +1,27 @@
+---
+layout: page
+title: Spark Structured Streaming MQTT
+description: Spark Structured Streaming MQTT
+group: nav-right
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+{% include JB/setup %}
+
http://git-wip-us.apache.org/repos/asf/bahir-website/blob/f8d35698/site/docs/spark/current/spark-streaming-akka.template
----------------------------------------------------------------------
diff --git a/site/docs/spark/current/spark-streaming-akka.template b/site/docs/spark/current/spark-streaming-akka.template
new file mode 100644
index 0000000..78b7d14
--- /dev/null
+++ b/site/docs/spark/current/spark-streaming-akka.template
@@ -0,0 +1,26 @@
+---
+layout: page
+title: Spark Streaming Akka
+description: Spark Streaming Akka
+group: nav-right
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+{% include JB/setup %}
http://git-wip-us.apache.org/repos/asf/bahir-website/blob/f8d35698/site/docs/spark/current/spark-streaming-mqtt.template
----------------------------------------------------------------------
diff --git a/site/docs/spark/current/spark-streaming-mqtt.template b/site/docs/spark/current/spark-streaming-mqtt.template
new file mode 100644
index 0000000..fd08baa
--- /dev/null
+++ b/site/docs/spark/current/spark-streaming-mqtt.template
@@ -0,0 +1,27 @@
+---
+layout: page
+title: Spark Structured Streaming MQTT
+description: Spark Structured Streaming MQTT
+group: nav-right
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+{% include JB/setup %}
+
http://git-wip-us.apache.org/repos/asf/bahir-website/blob/f8d35698/site/docs/spark/current/spark-streaming-twitter.template
----------------------------------------------------------------------
diff --git a/site/docs/spark/current/spark-streaming-twitter.template b/site/docs/spark/current/spark-streaming-twitter.template
new file mode 100644
index 0000000..9a97ced
--- /dev/null
+++ b/site/docs/spark/current/spark-streaming-twitter.template
@@ -0,0 +1,26 @@
+---
+layout: page
+title: Spark Streaming Twitter
+description: Spark Streaming Twitter
+group: nav-right
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+{% include JB/setup %}
http://git-wip-us.apache.org/repos/asf/bahir-website/blob/f8d35698/site/docs/spark/current/spark-streaming-zeromq.template
----------------------------------------------------------------------
diff --git a/site/docs/spark/current/spark-streaming-zeromq.template b/site/docs/spark/current/spark-streaming-zeromq.template
new file mode 100644
index 0000000..6a6a02a
--- /dev/null
+++ b/site/docs/spark/current/spark-streaming-zeromq.template
@@ -0,0 +1,26 @@
+---
+layout: page
+title: Spark Streaming ZeroMQ
+description: Spark Streaming ZeroMQ
+group: nav-right
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+{% include JB/setup %}
http://git-wip-us.apache.org/repos/asf/bahir-website/blob/f8d35698/update-doc.sh
----------------------------------------------------------------------
diff --git a/update-doc.sh b/update-doc.sh
new file mode 100755
index 0000000..2373215
--- /dev/null
+++ b/update-doc.sh
@@ -0,0 +1,61 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to you under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+set -e
+
+BASE_DIR=$(pwd)
+WEBSITE_DOC_DIR=$BASE_DIR/site/docs/spark/current
+BAHIR_SOURCE_DIR=$BASE_DIR/target/bahir
+
+function checkout_code {
+ # Checkout code
+ rm -rf target
+ mkdir target
+ cd target
+ rm -rf bahir
+ git clone https://git-wip-us.apache.org/repos/asf/bahir.git
+ cd bahir
+ git checkout $GIT_REF
+ git_hash=`git rev-parse --short HEAD`
+ echo "Checked out Bahir git hash $git_hash"
+
+ git clean -d -f -x
+
+ cd "$BASE_DIR" #return to base dir
+}
+
+checkout_code
+
+rm $WEBSITE_DOC_DIR/spark*.md
+
+cp $WEBSITE_DOC_DIR/spark-sql-streaming-mqtt.template $WEBSITE_DOC_DIR/spark-sql-streaming-mqtt.md
+cat $BAHIR_SOURCE_DIR/sql-streaming-mqtt/README.md >> $WEBSITE_DOC_DIR/spark-sql-streaming-mqtt.md
+
+cp $WEBSITE_DOC_DIR/spark-streaming-akka.template $WEBSITE_DOC_DIR/spark-streaming-akka.md
+cat $BAHIR_SOURCE_DIR/streaming-akka/README.md >> $WEBSITE_DOC_DIR/spark-streaming-akka.md
+
+cp $WEBSITE_DOC_DIR/spark-streaming-mqtt.template $WEBSITE_DOC_DIR/spark-streaming-mqtt.md
+cat $BAHIR_SOURCE_DIR/streaming-mqtt/README.md >> $WEBSITE_DOC_DIR/spark-streaming-mqtt.md
+
+cp $WEBSITE_DOC_DIR/spark-streaming-twitter.template $WEBSITE_DOC_DIR/spark-streaming-twitter.md
+cat $BAHIR_SOURCE_DIR/streaming-twitter/README.md >> $WEBSITE_DOC_DIR/spark-streaming-twitter.md
+
+cp $WEBSITE_DOC_DIR/spark-streaming-zeromq.template $WEBSITE_DOC_DIR/spark-streaming-zeromq.md
+cat $BAHIR_SOURCE_DIR/streaming-zeromq/README.md >> $WEBSITE_DOC_DIR/spark-streaming-zeromq.md
+
+set +e