You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by zh...@apache.org on 2022/07/27 14:45:49 UTC
[dolphinscheduler] 07/09: [python] Correct some docs for python api (#10772)
This is an automated email from the ASF dual-hosted git repository.
zhongjiajie pushed a commit to branch 3.0.0-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit 6f5ea6e41170f383072a6c13f7a1ecf29aa20d35
Author: Jiajie Zhong <zh...@hotmail.com>
AuthorDate: Tue Jul 5 18:34:31 2022 +0800
[python] Correct some docs for python api (#10772)
* Remove add leading `$` from command
* Add copy button from code block
* Using `python -m pip` to install package avoid error
(cherry picked from commit b4eee7edb6ef0a5fa8c27335067eccb0bb06a10f)
---
.../pydolphinscheduler/DEVELOP.md | 4 +-
.../pydolphinscheduler/README.md | 6 +-
.../pydolphinscheduler/docs/source/conf.py | 1 +
.../pydolphinscheduler/docs/source/config.rst | 71 +++++++++++-----------
.../docs/source/howto/remote-submit.rst | 6 +-
.../pydolphinscheduler/docs/source/start.rst | 49 +++++++++------
.../pydolphinscheduler/setup.py | 1 +
7 files changed, 75 insertions(+), 63 deletions(-)
diff --git a/dolphinscheduler-python/pydolphinscheduler/DEVELOP.md b/dolphinscheduler-python/pydolphinscheduler/DEVELOP.md
index beb95ad2b9..bdd0416675 100644
--- a/dolphinscheduler-python/pydolphinscheduler/DEVELOP.md
+++ b/dolphinscheduler-python/pydolphinscheduler/DEVELOP.md
@@ -34,7 +34,7 @@ Now, we should install all dependence to make sure we could run test or check co
```shell
cd dolphinscheduler/dolphinscheduler-python/pydolphinscheduler
-pip install .[dev]
+python -m pip install .[dev]
```
Next, we have to open pydolphinscheduler project in you editor. We recommend you use [pycharm][pycharm]
@@ -154,7 +154,7 @@ GitHub. You may locally ensure docs could be built suceessfully in case the fail
To build docs locally, install sphinx and related python modules first via:
```shell
-pip install '.[doc]'
+python -m pip install '.[doc]'
```
Then
diff --git a/dolphinscheduler-python/pydolphinscheduler/README.md b/dolphinscheduler-python/pydolphinscheduler/README.md
index 32a7ff6839..316d60406a 100644
--- a/dolphinscheduler-python/pydolphinscheduler/README.md
+++ b/dolphinscheduler-python/pydolphinscheduler/README.md
@@ -38,11 +38,11 @@ your workflow by python code, aka workflow-as-codes.
```shell
# Install
-$ pip install apache-dolphinscheduler
+python -m pip install apache-dolphinscheduler
# Verify installation is successful, it will show the version of apache-dolphinscheduler, here we use 0.1.0 as example
-$ pydolphinscheduler version
-0.1.0
+pydolphinscheduler version
+# 0.1.0
```
Here we show you how to install and run a simple example of pydolphinscheduler
diff --git a/dolphinscheduler-python/pydolphinscheduler/docs/source/conf.py b/dolphinscheduler-python/pydolphinscheduler/docs/source/conf.py
index efede5c299..b162e0cc50 100644
--- a/dolphinscheduler-python/pydolphinscheduler/docs/source/conf.py
+++ b/dolphinscheduler-python/pydolphinscheduler/docs/source/conf.py
@@ -59,6 +59,7 @@ extensions = [
"sphinx_click.ext",
# Add inline tabbed content
"sphinx_inline_tabs",
+ "sphinx_copybutton",
]
# Add any paths that contain templates here, relative to this directory.
diff --git a/dolphinscheduler-python/pydolphinscheduler/docs/source/config.rst b/dolphinscheduler-python/pydolphinscheduler/docs/source/config.rst
index 2b804d0c62..feb147a92e 100644
--- a/dolphinscheduler-python/pydolphinscheduler/docs/source/config.rst
+++ b/dolphinscheduler-python/pydolphinscheduler/docs/source/config.rst
@@ -47,10 +47,10 @@ how to change them by Bash.
.. code-block:: bash
# Modify Java Gateway Address
- $ export PYDS_JAVA_GATEWAY_ADDRESS="192.168.1.1"
+ export PYDS_JAVA_GATEWAY_ADDRESS="192.168.1.1"
# Modify Workflow Default User
- $ export PYDS_WORKFLOW_USER="custom-user"
+ export PYDS_WORKFLOW_USER="custom-user"
After executing the commands above, both ``PYDS_JAVA_GATEWAY_ADDRESS`` and ``PYDS_WORKFLOW_USER`` will be changed.
The next time you execute and submit your workflow, it will submit to host `192.168.1.1`, and with workflow's user
@@ -136,15 +136,15 @@ integrated built-in configurations in its package, but you could also export it
.. code-block:: bash
- $ pydolphinscheduler config --init
+ pydolphinscheduler config --init
And it will create a new YAML file in the path `~/pydolphinscheduler/config.yaml` by default. If you want to export
it to another path, you should set `PYDS_HOME` before you run command :code:`pydolphinscheduler config --init`.
.. code-block:: bash
- $ export PYDS_HOME=<CUSTOM_PATH>
- $ pydolphinscheduler config --init
+ export PYDS_HOME=<CUSTOM_PATH>
+ pydolphinscheduler config --init
After that, your configuration file will export into `<CUSTOM_PATH>/config.yaml` instead of the default path.
@@ -161,36 +161,37 @@ node of the configuration but could get the parent configuration, there are simp
.. code-block:: bash
- # Get single configuration in the leaf node
- $ pydolphinscheduler config --get java_gateway.address
- The configuration query as below:
-
- java_gateway.address = 127.0.0.1
-
- # Get multiple configuration in the leaf node
- $ pydolphinscheduler config --get java_gateway.address --get java_gateway.port
- The configuration query as below:
-
- java_gateway.address = 127.0.0.1
- java_gateway.port = 25333
-
- # Get parent configuration which contain multiple leaf nodes
- $ pydolphinscheduler config --get java_gateway
- The configuration query as below:
-
- java_gateway = ordereddict([('address', '127.0.0.1'), ('port', 25333), ('auto_convert', True)])
-
- # Set single configuration
- $ pydolphinscheduler config --set java_gateway.address 192.168.1.1
- Set configuration done.
-
- # Set multiple configuration
- $ pydolphinscheduler config --set java_gateway.address 192.168.1.1 --set java_gateway.port 25334
- Set configuration done.
-
- # Set configuration not in leaf node will fail
- $ pydolphinscheduler config --set java_gateway 192.168.1.1,25334,True
- Raise error.
+ # Get single configuration in the leaf node,
+ # The output look like below:
+ # java_gateway.address = 127.0.0.1
+ pydolphinscheduler config --get java_gateway.address
+
+ # Get multiple configuration in the leaf node,
+ # The output look like below:
+ # java_gateway.address = 127.0.0.1
+ # java_gateway.port = 25333
+ pydolphinscheduler config --get java_gateway.address --get java_gateway.port
+
+
+ # Get parent configuration which contain multiple leaf nodes,
+ # The output look like below:
+ # java_gateway = ordereddict([('address', '127.0.0.1'), ('port', 25333), ('auto_convert', True)])
+ pydolphinscheduler config --get java_gateway
+
+ # Set single configuration,
+ # The output look like below:
+ # Set configuration done.
+ pydolphinscheduler config --set java_gateway.address 192.168.1.1
+
+ # Set multiple configuration
+ # The output look like below:
+ # Set configuration done.
+ pydolphinscheduler config --set java_gateway.address 192.168.1.1 --set java_gateway.port 25334
+
+ # Set configuration not in leaf node will fail
+ # The output look like below:
+ # Raise error.
+ pydolphinscheduler config --set java_gateway 192.168.1.1,25334,True
For more information about our CLI, you could see document :doc:`cli`.
diff --git a/dolphinscheduler-python/pydolphinscheduler/docs/source/howto/remote-submit.rst b/dolphinscheduler-python/pydolphinscheduler/docs/source/howto/remote-submit.rst
index 43d03cce38..b7efdf4fc0 100644
--- a/dolphinscheduler-python/pydolphinscheduler/docs/source/howto/remote-submit.rst
+++ b/dolphinscheduler-python/pydolphinscheduler/docs/source/howto/remote-submit.rst
@@ -28,7 +28,7 @@ Export Configuration File
.. code-block:: bash
- $ pydolphinscheduler config --init
+ pydolphinscheduler config --init
your could find more detail in :ref:`configuration exporting <config:export configuration file>`
@@ -37,7 +37,7 @@ Run API Server in Other Host
.. code-block:: bash
- $ pydolphinscheduler config --set java_gateway.address <your-api-server-ip-or-hostname>
+ pydolphinscheduler config --set java_gateway.address <your-api-server-ip-or-hostname>
your could find more detail in :ref:`configuration setting <config:change configuration>`
@@ -46,6 +46,6 @@ Run API Server in Other Port
.. code-block:: bash
- $ pydolphinscheduler config --set java_gateway.port <your-python-gateway-service-port>
+ pydolphinscheduler config --set java_gateway.port <your-python-gateway-service-port>
your could find more detail in :ref:`configuration setting <config:change configuration>`
diff --git a/dolphinscheduler-python/pydolphinscheduler/docs/source/start.rst b/dolphinscheduler-python/pydolphinscheduler/docs/source/start.rst
index 6663c085e9..a483b0d59f 100644
--- a/dolphinscheduler-python/pydolphinscheduler/docs/source/start.rst
+++ b/dolphinscheduler-python/pydolphinscheduler/docs/source/start.rst
@@ -41,8 +41,9 @@ without error(here is a example after Python 3.8.7 installed)
.. code-block:: bash
- $ python --version
- Python 3.8.7
+ python --version
+
+Will see detail of Python version, such as *Python 3.8.7*
Installing PyDolphinScheduler
-----------------------------
@@ -52,7 +53,7 @@ After Python is already installed on your machine following section
.. code-block:: bash
- $ pip install apache-dolphinscheduler
+ python -m pip install apache-dolphinscheduler
The latest version of *PyDolphinScheduler* would be installed after you run above
command in your terminal. You could go and `start Python Gateway Service`_ to finish
@@ -60,8 +61,16 @@ the prepare, and then go to :doc:`tutorial` to make your hand dirty. But if you
want to install the unreleased version of *PyDolphinScheduler*, you could go and see
section `installing PyDolphinScheduler in dev`_ for more detail.
-Installing PyDolphinScheduler In Dev
-------------------------------------
+.. note::
+
+ Currently, we released multiple pre-release package in PyPI, you can see all released package
+ including pre-release in `release history <https://pypi.org/project/apache-dolphinscheduler/#history>`_.
+ You can fix the the package version if you want to install pre-release package, for example if
+ you want to install version `3.0.0-beta-2` package, you can run command
+ :code:`python -m pip install apache-dolphinscheduler==3.0.0b2`.
+
+Installing PyDolphinScheduler In DEV Branch
+-------------------------------------------
Because the project is developing and some of the features still not release.
If you want to try some thing unreleased you could install from the source code
@@ -69,10 +78,10 @@ which we hold in GitHub
.. code-block:: bash
- # Clone Apache DolphinScheduler repository
- $ git clone git@github.com:apache/dolphinscheduler.git
- # Install PyDolphinScheduler in develop mode
- $ cd dolphinscheduler-python/pydolphinscheduler && pip install -e .
+ # Clone Apache DolphinScheduler repository
+ git clone git@github.com:apache/dolphinscheduler.git
+ # Install PyDolphinScheduler in develop mode
+ cd dolphinscheduler-python/pydolphinscheduler && python -m pip install -e .
After you installed *PyDolphinScheduler*, please remember `start Python Gateway Service`_
which waiting for *PyDolphinScheduler*'s workflow definition require.
@@ -88,18 +97,18 @@ go `install Apache DolphinScheduler`_ for more detail
.. code-block:: bash
- # Start DolphinScheduler api-server which including python gateway service
- $ ./bin/dolphinscheduler-daemon.sh start api-server
+ # Start DolphinScheduler api-server which including python gateway service
+ ./bin/dolphinscheduler-daemon.sh start api-server
To check whether the server is alive or not, you could run :code:`jps`. And
the server is health if keyword `ApiApplicationServer` in the console.
.. code-block:: bash
- $ jps
- ....
- 201472 ApiApplicationServer
- ....
+ jps
+ # ....
+ # 201472 ApiApplicationServer
+ # ....
.. note::
@@ -115,23 +124,23 @@ single bash command to get it
.. code-block:: bash
- $ wget https://raw.githubusercontent.com/apache/dolphinscheduler/dev/dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/examples/tutorial.py
+ wget https://raw.githubusercontent.com/apache/dolphinscheduler/dev/dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/examples/tutorial.py
or you could copy-paste the content from `tutorial source code`_. And then you could run the example in your
terminal
.. code-block:: bash
- $ python tutorial.py
+ python tutorial.py
If you want to submit your workflow to a remote API server, which means that your workflow script is different
from the API server, you should first change pydolphinscheduler configuration and then submit the workflow script
.. code-block:: bash
- $ pydolphinscheduler config --init
- $ pydolphinscheduler config --set java_gateway.address <your-api-server-ip-or-hostname>
- $ python tutorial.py
+ pydolphinscheduler config --init
+ pydolphinscheduler config --set java_gateway.address <YOUR-API-SERVER-IP-OR-HOSTNAME>
+ python tutorial.py
.. note::
diff --git a/dolphinscheduler-python/pydolphinscheduler/setup.py b/dolphinscheduler-python/pydolphinscheduler/setup.py
index ee48de9b53..16f11a2ec2 100644
--- a/dolphinscheduler-python/pydolphinscheduler/setup.py
+++ b/dolphinscheduler-python/pydolphinscheduler/setup.py
@@ -52,6 +52,7 @@ doc = [
"sphinx_rtd_theme>=1.0",
"sphinx-click>=3.0",
"sphinx-inline-tabs",
+ "sphinx-copybutton>=0.4.0",
]
test = [