You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@streampipes.apache.org by bo...@apache.org on 2022/11/21 09:27:18 UTC

[streampipes] branch dev updated: Streampipes 634 - ADD MkDocs Documentation (#143)

This is an automated email from the ASF dual-hosted git repository.

bossenti pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/streampipes.git


The following commit(s) were added to refs/heads/dev by this push:
     new fbbe78677 Streampipes 634 - ADD MkDocs Documentation  (#143)
fbbe78677 is described below

commit fbbe78677470fead5c41f2effe15a255be73c9ac
Author: Cedric Kulbach <42...@users.noreply.github.com>
AuthorDate: Mon Nov 21 10:27:13 2022 +0100

    Streampipes 634 - ADD MkDocs Documentation  (#143)
    
    * First Draft MkDocs
    
    * Refactor
    
    * Add Documentation
    
    * CLEANING
    
    * Add github batch
    
    * [STREAMPIPES-634] fix emoji rendering
    
    * [STREAMPIPES-634] add missing license headers
    
    * [STREAMPIPES-634] add missing README part
    
    * [STREAMPIPES-364] some minor improvements
    
    Co-authored-by: Cedric Kulbach <ce...@googlemail.com>
    Co-authored-by: bossenti <bo...@posteo.de>
---
 .../docs/getting-started/installation.md           |  41 +++++++++
 .../docs/getting-started/quickstart.md             |  68 ++++++++++++++
 streampipes-client-python/docs/index.md            | 102 +++++++++++++++++++++
 streampipes-client-python/docs/overrides/main.html |  47 ++++++++++
 .../docs/overrides/partials/footer.html            |  47 ++++++++++
 .../docs/scripts/gen_ref_pages.py                  |  50 ++++++++++
 .../docs/stylesheets/extra.css                     |  21 +++++
 streampipes-client-python/mkdocs.yml               |  81 ++++++++++++++++
 streampipes-client-python/setup.py                 |   6 +-
 9 files changed, 462 insertions(+), 1 deletion(-)

diff --git a/streampipes-client-python/docs/getting-started/installation.md b/streampipes-client-python/docs/getting-started/installation.md
new file mode 100644
index 000000000..b072c5b21
--- /dev/null
+++ b/streampipes-client-python/docs/getting-started/installation.md
@@ -0,0 +1,41 @@
+<!--
+  ~ Licensed to the Apache Software Foundation (ASF) under one or more
+  ~ contributor license agreements.  See the NOTICE file distributed with
+  ~ this work for additional information regarding copyright ownership.
+  ~ The ASF licenses this file to You under the Apache License, Version 2.0
+  ~ (the "License"); you may not use this file except in compliance with
+  ~ the License.  You may obtain a copy of the License at
+  ~
+  ~    http://www.apache.org/licenses/LICENSE-2.0
+  ~
+  ~ Unless required by applicable law or agreed to in writing, software
+  ~ distributed under the License is distributed on an "AS IS" BASIS,
+  ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  ~ See the License for the specific language governing permissions and
+  ~ limitations under the License.
+  ~
+-->
+
+# πŸ“š Installation
+
+The StreamPipes Python Client is meant to work with Python 3.8 and above. Installation can be done via `pip`:
+You can install the latest development version from GitHub, as so:
+
+```bash
+pip install <NEEDS TO BE CLARIFIED>
+```
+
+---
+# πŸ‘¨β€πŸ’» Contributing
+*Before opening a pull request*, review the [Get Involved](https://streampipes.apache.org/getinvolved.html) page.
+It lists information that is required for contributing to StreamPipes.
+
+When you contribute code, you affirm that the contribution is your original work and that you
+license the work to the project under the project's open source license. Whether or not you
+state this explicitly, by submitting any copyrighted material via pull request, email, or
+other means you agree to license the material under the project's open source license and
+warrant that you have the legal authority to do so.
+
+For more information on how to start with the Python client, click [here](https://github.com/apache/streampipes/blob/dev/streampipes-client-python/DEVELOPMENT.md).
+
+---
\ No newline at end of file
diff --git a/streampipes-client-python/docs/getting-started/quickstart.md b/streampipes-client-python/docs/getting-started/quickstart.md
new file mode 100644
index 000000000..078fa7ffd
--- /dev/null
+++ b/streampipes-client-python/docs/getting-started/quickstart.md
@@ -0,0 +1,68 @@
+<!--
+  ~ Licensed to the Apache Software Foundation (ASF) under one or more
+  ~ contributor license agreements.  See the NOTICE file distributed with
+  ~ this work for additional information regarding copyright ownership.
+  ~ The ASF licenses this file to You under the Apache License, Version 2.0
+  ~ (the "License"); you may not use this file except in compliance with
+  ~ the License.  You may obtain a copy of the License at
+  ~
+  ~    http://www.apache.org/licenses/LICENSE-2.0
+  ~
+  ~ Unless required by applicable law or agreed to in writing, software
+  ~ distributed under the License is distributed on an "AS IS" BASIS,
+  ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  ~ See the License for the specific language governing permissions and
+  ~ limitations under the License.
+  ~
+-->
+
+# ⚑️ Quickstart
+
+As a quick example, we demonstrate how to set up and configure a StreamPipes client.
+In addition, we will get the available data lake measures out of StreamPipes.
+
+```python
+>>> from streampipes_client.client import StreamPipesClient
+>>> from streampipes_client.client.client_config import StreamPipesClientConfig
+>>> from streampipes_client.client.credential_provider import StreamPipesApiKeyCredentials
+
+>>> config = StreamPipesClientConfig(
+...    credential_provider=StreamPipesApiKeyCredentials(
+...         username="test@streampipes.apache.org",
+...         api_key="DEMO-KEY",
+...         ),
+...     host_address="localhost",
+...     http_disabled=True,
+...     port=80                  
+...)
+
+>>> client = StreamPipesClient(client_config=config)
+
+# get all available datat lake measures
+>>> measures = client.dataLakeMeasureApi.all()
+
+# get amount of retrieved measures
+>>> len(measures)
+1
+
+# inspect the data lake measures as pandas dataframe
+>>> measures.to_pandas()
+    measure_name timestamp_field  ... pipeline_is_running num_event_properties
+0           test   s0::timestamp  ...               False                    2
+[1 rows x 6 columns]
+```
+<br>
+Alternatively, you can provide your credentials via environment variables.
+Simply define your credential provider as follows:
+
+```python
+>>> from streampipes_client.client.credential_provider import StreamPipesApiKeyCredentials
+
+StreamPipesApiKeyCredentials.from_env(username_env="USER", api_key_env="API-KEY")
+```
+<br>
+
+`username` is always the username that is used to log in into StreamPipes. <br>
+The `api_key` can be generated within the UI as demonstrated below:
+
+![Howto API Key](https://raw.githubusercontent.com/apache/incubator-streampipes/dev/streampipes-client-python/docs/img/how-to-get-api-key.gif)
\ No newline at end of file
diff --git a/streampipes-client-python/docs/index.md b/streampipes-client-python/docs/index.md
new file mode 100644
index 000000000..74d3824dc
--- /dev/null
+++ b/streampipes-client-python/docs/index.md
@@ -0,0 +1,102 @@
+<!--
+  ~ Licensed to the Apache Software Foundation (ASF) under one or more
+  ~ contributor license agreements.  See the NOTICE file distributed with
+  ~ this work for additional information regarding copyright ownership.
+  ~ The ASF licenses this file to You under the Apache License, Version 2.0
+  ~ (the "License"); you may not use this file except in compliance with
+  ~ the License.  You may obtain a copy of the License at
+  ~
+  ~    http://www.apache.org/licenses/LICENSE-2.0
+  ~
+  ~ Unless required by applicable law or agreed to in writing, software
+  ~ distributed under the License is distributed on an "AS IS" BASIS,
+  ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  ~ See the License for the specific language governing permissions and
+  ~ limitations under the License.
+  ~
+-->
+
+
+<h1 align="center">
+  <br>
+   <img src="https://raw.githubusercontent.com/apache/incubator-streampipes/dev/streampipes-client-python/docs/img/streampipes-python.png"
+   alt="StreamPipes Logo with Python" title="Apache StreamPipes Logo with Python" width="75%"/>
+  <br>
+</h1>
+<h4 align="center"><a href="[StreamPipes](https://github.com/apache/incubator-streampipes)">StreamPipes</a> is a self-service (Industrial) IoT toolbox to enable non-technical users to connect , analyze and explore IoT data streams.</h4>
+
+<br>
+<h3 align="center">Apache StreamPipes client for Python</h3>
+
+<p align="center"> Apache StreamPipes meets Python! We are working highly motivated on a Python-based client to interact with StreamPipes.
+In this way, we would like to unite the power of StreamPipes to easily connect to and read different data sources, especially in the IoT domain,
+and the amazing universe of data analytics libraries in Python. </p>
+
+---
+
+<br>
+
+** ❗❗❗IMPORTANT ❗❗❗**
+<br>
+<br>
+**The current version of this Python client is still in alpha phase at best.**
+<br>
+**This means that it is still heavily under development, which may result in frequent and extensive API changes, unstable behavior, etc.**
+<br>
+**Please consider it only as a sneak preview.**
+<br>
+<br>
+**❗❗❗ IMPORTANT ❗❗❗**
+
+<br>
+
+## ⚑️ Quickstart
+
+As a quick example, we demonstrate how to set up and configure a StreamPipes client.
+In addition, we will get the available data lake measures out of StreamPipes.
+
+```python
+>>> from streampipes_client.client import StreamPipesClient
+>>> from streampipes_client.client.client_config import StreamPipesClientConfig
+>>> from streampipes_client.client.credential_provider import StreamPipesApiKeyCredentials
+
+>>> config = StreamPipesClientConfig(
+...    credential_provider=StreamPipesApiKeyCredentials(
+...         username="test@streampipes.apache.org",
+...         api_key="DEMO-KEY",
+...         ),
+...     host_address="localhost",
+...     http_disabled=True,
+...     port=80                  
+...)
+
+>>> client = StreamPipesClient(client_config=config)
+
+# get all available datat lake measures
+>>> measures = client.dataLakeMeasureApi.all()
+
+# get amount of retrieved measures
+>>> len(measures)
+1
+
+# inspect the data lake measures as pandas dataframe
+>>> measures.to_pandas()
+    measure_name timestamp_field  ... pipeline_is_running num_event_properties
+0           test   s0::timestamp  ...               False                    2
+[1 rows x 6 columns]
+```
+<br>
+Alternatively, you can provide your credentials via environment variables.
+Simply define your credential provider as follows:
+
+```python
+>>> from streampipes_client.client.credential_provider import StreamPipesApiKeyCredentials
+
+StreamPipesApiKeyCredentials.from_env(username_env="USER", api_key_env="API-KEY")
+```
+<br>
+
+`username` is always the username that is used to log in into StreamPipes. <br>
+The `api_key` can be generated within the UI as demonstrated below:
+
+![Howto API Key](https://raw.githubusercontent.com/apache/incubator-streampipes/dev/streampipes-client-python/docs/img/how-to-get-api-key.gif)
\ No newline at end of file
diff --git a/streampipes-client-python/docs/overrides/main.html b/streampipes-client-python/docs/overrides/main.html
new file mode 100644
index 000000000..c49308e77
--- /dev/null
+++ b/streampipes-client-python/docs/overrides/main.html
@@ -0,0 +1,47 @@
+<!--
+  ~ Licensed to the Apache Software Foundation (ASF) under one or more
+  ~ contributor license agreements.  See the NOTICE file distributed with
+  ~ this work for additional information regarding copyright ownership.
+  ~ The ASF licenses this file to You under the Apache License, Version 2.0
+  ~ (the "License"); you may not use this file except in compliance with
+  ~ the License.  You may obtain a copy of the License at
+  ~
+  ~    http://www.apache.org/licenses/LICENSE-2.0
+  ~
+  ~ Unless required by applicable law or agreed to in writing, software
+  ~ distributed under the License is distributed on an "AS IS" BASIS,
+  ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  ~ See the License for the specific language governing permissions and
+  ~ limitations under the License.
+  ~
+-->
+
+<!-- Elements added to main will be displayed on all pages -->
+{% extends "base.html" %}
+
+<!-- More minimal footer -->
+{% block footer %}
+<footer class="md-footer">
+    <div class="md-footer-meta md-typeset">
+        <div class="md-footer-meta__inner md-grid">
+            <!-- Copyright and theme information -->
+            <div class="md-footer-copyright">
+                {% if config.copyright %}
+                <div class="md-footer-copyright__highlight">
+                    {{ config.copyright }}
+                </div>
+                {% endif %}
+                powered by
+                <a href="https://www.mkdocs.org" title="MkDocs">MkDocs</a>
+                and
+                <a href="https://squidfunk.github.io/mkdocs-material/"
+                   title="Material for MkDocs">
+                    Material for MkDocs</a>
+            </div>
+            {% block social %}
+            {% include "partials/social.html" %}
+            {% endblock %}
+        </div>
+    </div>
+</footer>
+{% endblock %}
\ No newline at end of file
diff --git a/streampipes-client-python/docs/overrides/partials/footer.html b/streampipes-client-python/docs/overrides/partials/footer.html
new file mode 100644
index 000000000..6e0781595
--- /dev/null
+++ b/streampipes-client-python/docs/overrides/partials/footer.html
@@ -0,0 +1,47 @@
+<!--
+  ~ Licensed to the Apache Software Foundation (ASF) under one or more
+  ~ contributor license agreements.  See the NOTICE file distributed with
+  ~ this work for additional information regarding copyright ownership.
+  ~ The ASF licenses this file to You under the Apache License, Version 2.0
+  ~ (the "License"); you may not use this file except in compliance with
+  ~ the License.  You may obtain a copy of the License at
+  ~
+  ~    http://www.apache.org/licenses/LICENSE-2.0
+  ~
+  ~ Unless required by applicable law or agreed to in writing, software
+  ~ distributed under the License is distributed on an "AS IS" BASIS,
+  ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  ~ See the License for the specific language governing permissions and
+  ~ limitations under the License.
+  ~
+-->
+{% import "partials/language.html" as lang with context %}
+
+<!-- Application footer -->
+<footer class="md-footer">
+
+  <!-- Further information -->
+  <div class="md-footer-meta md-typeset">
+    <div class="md-footer-meta__inner md-grid">
+
+      <!-- Copyright and theme information -->
+      <div class="md-footer-copyright">
+        {% if config.copyright %}
+          <div class="md-footer-copyright__highlight">
+            {{ config.copyright }}
+          </div>
+        {% endif %}
+        Made with
+        <a
+          href="https://squidfunk.github.io/mkdocs-material/"
+          target="_blank" rel="noopener"
+        >
+          Material for MkDocs
+        </a>
+      </div>
+
+      <!-- Social links -->
+      {% include "partials/social.html" %}
+    </div>
+  </div>
+</footer>
diff --git a/streampipes-client-python/docs/scripts/gen_ref_pages.py b/streampipes-client-python/docs/scripts/gen_ref_pages.py
new file mode 100644
index 000000000..9d935d0a7
--- /dev/null
+++ b/streampipes-client-python/docs/scripts/gen_ref_pages.py
@@ -0,0 +1,50 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+from pathlib import Path
+
+import mkdocs_gen_files
+
+nav = mkdocs_gen_files.Nav()
+
+for path in sorted(Path("streampipes_client").rglob("*.py")):
+    module_path = path.relative_to("streampipes_client").with_suffix("")
+    doc_path = path.relative_to("streampipes_client").with_suffix(".md")
+    full_doc_path = Path("reference", doc_path)
+
+    parts = list(module_path.parts)
+
+    if parts[-1] == "__init__":
+        # parts = parts[:-1]
+        continue
+    elif parts[-1] == "__version__":
+        continue
+    elif parts[-1] == "__main__":
+        continue
+    elif parts[-1] == " ":
+        continue
+
+    nav[parts] = doc_path.as_posix()
+
+    with mkdocs_gen_files.open(full_doc_path, "w+") as fd:
+        identifier = ".".join(parts)
+        print(f"::: streampipes_client.{identifier}", file=fd)
+
+    mkdocs_gen_files.set_edit_path(full_doc_path, path)
+
+#with mkdocs_gen_files.open("reference/SUMMARY.md", "w+") as nav_file:
+#    nav_file.writelines(nav.build_literate_nav())
diff --git a/streampipes-client-python/docs/stylesheets/extra.css b/streampipes-client-python/docs/stylesheets/extra.css
new file mode 100644
index 000000000..de44469b9
--- /dev/null
+++ b/streampipes-client-python/docs/stylesheets/extra.css
@@ -0,0 +1,21 @@
+/* Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements.  See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. */
+
+:root > * {
+  --md-primary-fg-color:        #39B54A;
+  --md-primary-fg-color--light: #3BCD4A;
+  --md-accent-fg-color:         #1B1464;
+  --md-accent-fg-color--light:  #9F9C9C;
+}
\ No newline at end of file
diff --git a/streampipes-client-python/mkdocs.yml b/streampipes-client-python/mkdocs.yml
new file mode 100644
index 000000000..5bf0f9e83
--- /dev/null
+++ b/streampipes-client-python/mkdocs.yml
@@ -0,0 +1,81 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+site_name: Apache StreamPipes Python Client
+repo_url: https://github.com/apache/streampipes
+edit_uri: "https://github.com/apache/streampipes"
+
+copyright: "Apache License 2.0"
+
+theme:
+  name: material
+  logo: https://streampipes.apache.org/img/sp-logo-white-right.png
+  favicon: https://streampipes.apache.org/img/favicon/favicon.png
+  custom_dir: docs/overrides
+  features:
+    - navigation.tabs
+    - navigation.instant
+    - navigation.indexes
+    - navigation.tracking
+
+extra:
+  social:
+    - icon: fontawesome/brands/github-alt
+      link: https://github.com/apache/streampipes
+
+# Extensions
+markdown_extensions:
+  - admonition
+  - footnotes
+  - toc:
+      permalink: true
+      toc_depth: "1-3"
+  - pymdownx.details
+  - pymdownx.arithmatex:
+      generic: true
+  - pymdownx.details
+  - pymdownx.superfences
+  - pymdownx.highlight:
+        use_pygments: true
+        pygments_lang_class: true
+
+plugins:
+  - awesome-pages
+  - autorefs
+  - gen-files:
+      scripts:
+        - docs/scripts/gen_ref_pages.py
+  - literate-nav:
+      nav_file: SUMMARY.md
+  - mkdocstrings:
+      handlers:
+        python:
+          options:
+            docstring_style: numpy
+            docstring_section_style: spacy
+            show_source: False
+            merge_init_into_class: True
+
+extra_css:
+  - stylesheets/extra.css
+
+nav:
+    - Home: index.md
+    - Getting Started:
+        - Installation: getting-started/installation.md
+        - Quickstart: getting-started/quickstart.md
+    - Reference: reference/*
\ No newline at end of file
diff --git a/streampipes-client-python/setup.py b/streampipes-client-python/setup.py
index ee1d50c28..99a5ee2be 100644
--- a/streampipes-client-python/setup.py
+++ b/streampipes-client-python/setup.py
@@ -43,7 +43,7 @@ dev_packages = base_packages + [
     "interrogate>=1.5.0",
     "isort>=5.10.1",
     "mypy>=0.990",
-    "pandas-stubs>=1.5.1.221024",
+    "pandas-stubs>=1.2.0.62",
     "pre-commit>=2.20.0",
     "pytest>=7.2.0",
     "pytest-cov>=4.0.0",
@@ -53,8 +53,12 @@ dev_packages = base_packages + [
 
 docs_packages = [
     "mkdocs>=1.2.3",
+    "mkdocs-awesome-pages-plugin>=2.7.0",
     "mkdocs-material>=8.1.11",
     "mkdocstrings[python]>=0.19.0",
+    "pytkdocs[numpy-style]>=0.5.0",
+    "mkdocs-gen-files>=0.3.5",
+    "mkdocs-literate-nav>=0.4.1",
     "numpydoc>=1.2",
 ]