You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@streampipes.apache.org by bo...@apache.org on 2022/11/21 09:27:18 UTC
[streampipes] branch dev updated: Streampipes 634 - ADD MkDocs Documentation (#143)
This is an automated email from the ASF dual-hosted git repository.
bossenti pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/streampipes.git
The following commit(s) were added to refs/heads/dev by this push:
new fbbe78677 Streampipes 634 - ADD MkDocs Documentation (#143)
fbbe78677 is described below
commit fbbe78677470fead5c41f2effe15a255be73c9ac
Author: Cedric Kulbach <42...@users.noreply.github.com>
AuthorDate: Mon Nov 21 10:27:13 2022 +0100
Streampipes 634 - ADD MkDocs Documentation (#143)
* First Draft MkDocs
* Refactor
* Add Documentation
* CLEANING
* Add github batch
* [STREAMPIPES-634] fix emoji rendering
* [STREAMPIPES-634] add missing license headers
* [STREAMPIPES-634] add missing README part
* [STREAMPIPES-364] some minor improvements
Co-authored-by: Cedric Kulbach <ce...@googlemail.com>
Co-authored-by: bossenti <bo...@posteo.de>
---
.../docs/getting-started/installation.md | 41 +++++++++
.../docs/getting-started/quickstart.md | 68 ++++++++++++++
streampipes-client-python/docs/index.md | 102 +++++++++++++++++++++
streampipes-client-python/docs/overrides/main.html | 47 ++++++++++
.../docs/overrides/partials/footer.html | 47 ++++++++++
.../docs/scripts/gen_ref_pages.py | 50 ++++++++++
.../docs/stylesheets/extra.css | 21 +++++
streampipes-client-python/mkdocs.yml | 81 ++++++++++++++++
streampipes-client-python/setup.py | 6 +-
9 files changed, 462 insertions(+), 1 deletion(-)
diff --git a/streampipes-client-python/docs/getting-started/installation.md b/streampipes-client-python/docs/getting-started/installation.md
new file mode 100644
index 000000000..b072c5b21
--- /dev/null
+++ b/streampipes-client-python/docs/getting-started/installation.md
@@ -0,0 +1,41 @@
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ ~
+-->
+
+# π Installation
+
+The StreamPipes Python Client is meant to work with Python 3.8 and above. Installation can be done via `pip`:
+You can install the latest development version from GitHub, as so:
+
+```bash
+pip install <NEEDS TO BE CLARIFIED>
+```
+
+---
+# π¨βπ» Contributing
+*Before opening a pull request*, review the [Get Involved](https://streampipes.apache.org/getinvolved.html) page.
+It lists information that is required for contributing to StreamPipes.
+
+When you contribute code, you affirm that the contribution is your original work and that you
+license the work to the project under the project's open source license. Whether or not you
+state this explicitly, by submitting any copyrighted material via pull request, email, or
+other means you agree to license the material under the project's open source license and
+warrant that you have the legal authority to do so.
+
+For more information on how to start with the Python client, click [here](https://github.com/apache/streampipes/blob/dev/streampipes-client-python/DEVELOPMENT.md).
+
+---
\ No newline at end of file
diff --git a/streampipes-client-python/docs/getting-started/quickstart.md b/streampipes-client-python/docs/getting-started/quickstart.md
new file mode 100644
index 000000000..078fa7ffd
--- /dev/null
+++ b/streampipes-client-python/docs/getting-started/quickstart.md
@@ -0,0 +1,68 @@
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ ~
+-->
+
+# β‘οΈ Quickstart
+
+As a quick example, we demonstrate how to set up and configure a StreamPipes client.
+In addition, we will get the available data lake measures out of StreamPipes.
+
+```python
+>>> from streampipes_client.client import StreamPipesClient
+>>> from streampipes_client.client.client_config import StreamPipesClientConfig
+>>> from streampipes_client.client.credential_provider import StreamPipesApiKeyCredentials
+
+>>> config = StreamPipesClientConfig(
+... credential_provider=StreamPipesApiKeyCredentials(
+... username="test@streampipes.apache.org",
+... api_key="DEMO-KEY",
+... ),
+... host_address="localhost",
+... http_disabled=True,
+... port=80
+...)
+
+>>> client = StreamPipesClient(client_config=config)
+
+# get all available datat lake measures
+>>> measures = client.dataLakeMeasureApi.all()
+
+# get amount of retrieved measures
+>>> len(measures)
+1
+
+# inspect the data lake measures as pandas dataframe
+>>> measures.to_pandas()
+ measure_name timestamp_field ... pipeline_is_running num_event_properties
+0 test s0::timestamp ... False 2
+[1 rows x 6 columns]
+```
+<br>
+Alternatively, you can provide your credentials via environment variables.
+Simply define your credential provider as follows:
+
+```python
+>>> from streampipes_client.client.credential_provider import StreamPipesApiKeyCredentials
+
+StreamPipesApiKeyCredentials.from_env(username_env="USER", api_key_env="API-KEY")
+```
+<br>
+
+`username` is always the username that is used to log in into StreamPipes. <br>
+The `api_key` can be generated within the UI as demonstrated below:
+
+
\ No newline at end of file
diff --git a/streampipes-client-python/docs/index.md b/streampipes-client-python/docs/index.md
new file mode 100644
index 000000000..74d3824dc
--- /dev/null
+++ b/streampipes-client-python/docs/index.md
@@ -0,0 +1,102 @@
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ ~
+-->
+
+
+<h1 align="center">
+ <br>
+ <img src="https://raw.githubusercontent.com/apache/incubator-streampipes/dev/streampipes-client-python/docs/img/streampipes-python.png"
+ alt="StreamPipes Logo with Python" title="Apache StreamPipes Logo with Python" width="75%"/>
+ <br>
+</h1>
+<h4 align="center"><a href="[StreamPipes](https://github.com/apache/incubator-streampipes)">StreamPipes</a> is a self-service (Industrial) IoT toolbox to enable non-technical users to connect , analyze and explore IoT data streams.</h4>
+
+<br>
+<h3 align="center">Apache StreamPipes client for Python</h3>
+
+<p align="center"> Apache StreamPipes meets Python! We are working highly motivated on a Python-based client to interact with StreamPipes.
+In this way, we would like to unite the power of StreamPipes to easily connect to and read different data sources, especially in the IoT domain,
+and the amazing universe of data analytics libraries in Python. </p>
+
+---
+
+<br>
+
+** βββIMPORTANT βββ**
+<br>
+<br>
+**The current version of this Python client is still in alpha phase at best.**
+<br>
+**This means that it is still heavily under development, which may result in frequent and extensive API changes, unstable behavior, etc.**
+<br>
+**Please consider it only as a sneak preview.**
+<br>
+<br>
+**βββ IMPORTANT βββ**
+
+<br>
+
+## β‘οΈ Quickstart
+
+As a quick example, we demonstrate how to set up and configure a StreamPipes client.
+In addition, we will get the available data lake measures out of StreamPipes.
+
+```python
+>>> from streampipes_client.client import StreamPipesClient
+>>> from streampipes_client.client.client_config import StreamPipesClientConfig
+>>> from streampipes_client.client.credential_provider import StreamPipesApiKeyCredentials
+
+>>> config = StreamPipesClientConfig(
+... credential_provider=StreamPipesApiKeyCredentials(
+... username="test@streampipes.apache.org",
+... api_key="DEMO-KEY",
+... ),
+... host_address="localhost",
+... http_disabled=True,
+... port=80
+...)
+
+>>> client = StreamPipesClient(client_config=config)
+
+# get all available datat lake measures
+>>> measures = client.dataLakeMeasureApi.all()
+
+# get amount of retrieved measures
+>>> len(measures)
+1
+
+# inspect the data lake measures as pandas dataframe
+>>> measures.to_pandas()
+ measure_name timestamp_field ... pipeline_is_running num_event_properties
+0 test s0::timestamp ... False 2
+[1 rows x 6 columns]
+```
+<br>
+Alternatively, you can provide your credentials via environment variables.
+Simply define your credential provider as follows:
+
+```python
+>>> from streampipes_client.client.credential_provider import StreamPipesApiKeyCredentials
+
+StreamPipesApiKeyCredentials.from_env(username_env="USER", api_key_env="API-KEY")
+```
+<br>
+
+`username` is always the username that is used to log in into StreamPipes. <br>
+The `api_key` can be generated within the UI as demonstrated below:
+
+
\ No newline at end of file
diff --git a/streampipes-client-python/docs/overrides/main.html b/streampipes-client-python/docs/overrides/main.html
new file mode 100644
index 000000000..c49308e77
--- /dev/null
+++ b/streampipes-client-python/docs/overrides/main.html
@@ -0,0 +1,47 @@
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ ~
+-->
+
+<!-- Elements added to main will be displayed on all pages -->
+{% extends "base.html" %}
+
+<!-- More minimal footer -->
+{% block footer %}
+<footer class="md-footer">
+ <div class="md-footer-meta md-typeset">
+ <div class="md-footer-meta__inner md-grid">
+ <!-- Copyright and theme information -->
+ <div class="md-footer-copyright">
+ {% if config.copyright %}
+ <div class="md-footer-copyright__highlight">
+ {{ config.copyright }}
+ </div>
+ {% endif %}
+ powered by
+ <a href="https://www.mkdocs.org" title="MkDocs">MkDocs</a>
+ and
+ <a href="https://squidfunk.github.io/mkdocs-material/"
+ title="Material for MkDocs">
+ Material for MkDocs</a>
+ </div>
+ {% block social %}
+ {% include "partials/social.html" %}
+ {% endblock %}
+ </div>
+ </div>
+</footer>
+{% endblock %}
\ No newline at end of file
diff --git a/streampipes-client-python/docs/overrides/partials/footer.html b/streampipes-client-python/docs/overrides/partials/footer.html
new file mode 100644
index 000000000..6e0781595
--- /dev/null
+++ b/streampipes-client-python/docs/overrides/partials/footer.html
@@ -0,0 +1,47 @@
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ ~
+-->
+{% import "partials/language.html" as lang with context %}
+
+<!-- Application footer -->
+<footer class="md-footer">
+
+ <!-- Further information -->
+ <div class="md-footer-meta md-typeset">
+ <div class="md-footer-meta__inner md-grid">
+
+ <!-- Copyright and theme information -->
+ <div class="md-footer-copyright">
+ {% if config.copyright %}
+ <div class="md-footer-copyright__highlight">
+ {{ config.copyright }}
+ </div>
+ {% endif %}
+ Made with
+ <a
+ href="https://squidfunk.github.io/mkdocs-material/"
+ target="_blank" rel="noopener"
+ >
+ Material for MkDocs
+ </a>
+ </div>
+
+ <!-- Social links -->
+ {% include "partials/social.html" %}
+ </div>
+ </div>
+</footer>
diff --git a/streampipes-client-python/docs/scripts/gen_ref_pages.py b/streampipes-client-python/docs/scripts/gen_ref_pages.py
new file mode 100644
index 000000000..9d935d0a7
--- /dev/null
+++ b/streampipes-client-python/docs/scripts/gen_ref_pages.py
@@ -0,0 +1,50 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+from pathlib import Path
+
+import mkdocs_gen_files
+
+nav = mkdocs_gen_files.Nav()
+
+for path in sorted(Path("streampipes_client").rglob("*.py")):
+ module_path = path.relative_to("streampipes_client").with_suffix("")
+ doc_path = path.relative_to("streampipes_client").with_suffix(".md")
+ full_doc_path = Path("reference", doc_path)
+
+ parts = list(module_path.parts)
+
+ if parts[-1] == "__init__":
+ # parts = parts[:-1]
+ continue
+ elif parts[-1] == "__version__":
+ continue
+ elif parts[-1] == "__main__":
+ continue
+ elif parts[-1] == " ":
+ continue
+
+ nav[parts] = doc_path.as_posix()
+
+ with mkdocs_gen_files.open(full_doc_path, "w+") as fd:
+ identifier = ".".join(parts)
+ print(f"::: streampipes_client.{identifier}", file=fd)
+
+ mkdocs_gen_files.set_edit_path(full_doc_path, path)
+
+#with mkdocs_gen_files.open("reference/SUMMARY.md", "w+") as nav_file:
+# nav_file.writelines(nav.build_literate_nav())
diff --git a/streampipes-client-python/docs/stylesheets/extra.css b/streampipes-client-python/docs/stylesheets/extra.css
new file mode 100644
index 000000000..de44469b9
--- /dev/null
+++ b/streampipes-client-python/docs/stylesheets/extra.css
@@ -0,0 +1,21 @@
+/* Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. */
+
+:root > * {
+ --md-primary-fg-color: #39B54A;
+ --md-primary-fg-color--light: #3BCD4A;
+ --md-accent-fg-color: #1B1464;
+ --md-accent-fg-color--light: #9F9C9C;
+}
\ No newline at end of file
diff --git a/streampipes-client-python/mkdocs.yml b/streampipes-client-python/mkdocs.yml
new file mode 100644
index 000000000..5bf0f9e83
--- /dev/null
+++ b/streampipes-client-python/mkdocs.yml
@@ -0,0 +1,81 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+site_name: Apache StreamPipes Python Client
+repo_url: https://github.com/apache/streampipes
+edit_uri: "https://github.com/apache/streampipes"
+
+copyright: "Apache License 2.0"
+
+theme:
+ name: material
+ logo: https://streampipes.apache.org/img/sp-logo-white-right.png
+ favicon: https://streampipes.apache.org/img/favicon/favicon.png
+ custom_dir: docs/overrides
+ features:
+ - navigation.tabs
+ - navigation.instant
+ - navigation.indexes
+ - navigation.tracking
+
+extra:
+ social:
+ - icon: fontawesome/brands/github-alt
+ link: https://github.com/apache/streampipes
+
+# Extensions
+markdown_extensions:
+ - admonition
+ - footnotes
+ - toc:
+ permalink: true
+ toc_depth: "1-3"
+ - pymdownx.details
+ - pymdownx.arithmatex:
+ generic: true
+ - pymdownx.details
+ - pymdownx.superfences
+ - pymdownx.highlight:
+ use_pygments: true
+ pygments_lang_class: true
+
+plugins:
+ - awesome-pages
+ - autorefs
+ - gen-files:
+ scripts:
+ - docs/scripts/gen_ref_pages.py
+ - literate-nav:
+ nav_file: SUMMARY.md
+ - mkdocstrings:
+ handlers:
+ python:
+ options:
+ docstring_style: numpy
+ docstring_section_style: spacy
+ show_source: False
+ merge_init_into_class: True
+
+extra_css:
+ - stylesheets/extra.css
+
+nav:
+ - Home: index.md
+ - Getting Started:
+ - Installation: getting-started/installation.md
+ - Quickstart: getting-started/quickstart.md
+ - Reference: reference/*
\ No newline at end of file
diff --git a/streampipes-client-python/setup.py b/streampipes-client-python/setup.py
index ee1d50c28..99a5ee2be 100644
--- a/streampipes-client-python/setup.py
+++ b/streampipes-client-python/setup.py
@@ -43,7 +43,7 @@ dev_packages = base_packages + [
"interrogate>=1.5.0",
"isort>=5.10.1",
"mypy>=0.990",
- "pandas-stubs>=1.5.1.221024",
+ "pandas-stubs>=1.2.0.62",
"pre-commit>=2.20.0",
"pytest>=7.2.0",
"pytest-cov>=4.0.0",
@@ -53,8 +53,12 @@ dev_packages = base_packages + [
docs_packages = [
"mkdocs>=1.2.3",
+ "mkdocs-awesome-pages-plugin>=2.7.0",
"mkdocs-material>=8.1.11",
"mkdocstrings[python]>=0.19.0",
+ "pytkdocs[numpy-style]>=0.5.0",
+ "mkdocs-gen-files>=0.3.5",
+ "mkdocs-literate-nav>=0.4.1",
"numpydoc>=1.2",
]