You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/08/26 11:51:33 UTC
[GitHub] [flink] morsapaes commented on a change in pull request #13171: [FLINK-18927][python][doc] Add Debugging document in Python Table API
morsapaes commented on a change in pull request #13171:
URL: https://github.com/apache/flink/pull/13171#discussion_r477239105
##########
File path: docs/dev/python/user-guide/table/debugging.md
##########
@@ -0,0 +1,80 @@
+---
+title: "Debugging"
+nav-parent_id: python_tableapi
+nav-pos: 130
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+This page describes how to debug in PyFlink.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Logging Infos
+
+Python UDFs can log contextual and debug information via standard Python logging modules.
+
+{% highlight python %}
+
+@udf(input_types=[DataTypes.BIGINT(), DataTypes.BIGINT()], result_type=DataTypes.BIGINT())
+def add(i, j):
+ import logging
+ import sys
+ logging.basicConfig(stream=sys.stdout, level=logging.INFO, format="%(message)s")
+ logging.info("debug")
+ return i + j
+
+{% endhighlight %}
+
+## Watching Logs
+
+If the environment variable `FLINK_HOME` is set, logs will be written in the log directory under `FLINK_HOME`.
+Otherwise, logs will be placed in the directory of the PyFlink module. You can execute the following command to find
+the log directory of PyFlink module:
+
+{% highlight shell %}
+
+$ python -c "import pyflink;import os;print(os.path.dirname(os.path.abspath(pyflink.__file__))+'/log')"
+
+{% endhighlight %}
+
+## Debug Python UDF
+You can take use of the pydevd_pycharm tool of PyCharm to debug Python UDF.
Review comment:
```suggestion
You can make use of the [`pydevd_pycharm`](https://pypi.org/project/pydevd-pycharm/) tool of PyCharm to debug Python UDFs.
```
##########
File path: docs/dev/python/user-guide/table/debugging.md
##########
@@ -0,0 +1,80 @@
+---
+title: "Debugging"
+nav-parent_id: python_tableapi
+nav-pos: 130
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+This page describes how to debug in PyFlink.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Logging Infos
+
+Python UDFs can log contextual and debug information via standard Python logging modules.
+
+{% highlight python %}
+
+@udf(input_types=[DataTypes.BIGINT(), DataTypes.BIGINT()], result_type=DataTypes.BIGINT())
+def add(i, j):
+ import logging
+ import sys
+ logging.basicConfig(stream=sys.stdout, level=logging.INFO, format="%(message)s")
+ logging.info("debug")
+ return i + j
+
+{% endhighlight %}
+
+## Watching Logs
+
+If the environment variable `FLINK_HOME` is set, logs will be written in the log directory under `FLINK_HOME`.
+Otherwise, logs will be placed in the directory of the PyFlink module. You can execute the following command to find
+the log directory of PyFlink module:
+
+{% highlight shell %}
+
+$ python -c "import pyflink;import os;print(os.path.dirname(os.path.abspath(pyflink.__file__))+'/log')"
+
+{% endhighlight %}
+
+## Debug Python UDF
Review comment:
```suggestion
## Debugging Python UDFs
```
##########
File path: docs/dev/python/user-guide/table/debugging.md
##########
@@ -0,0 +1,80 @@
+---
+title: "Debugging"
+nav-parent_id: python_tableapi
+nav-pos: 130
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+This page describes how to debug in PyFlink.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Logging Infos
+
+Python UDFs can log contextual and debug information via standard Python logging modules.
+
+{% highlight python %}
+
+@udf(input_types=[DataTypes.BIGINT(), DataTypes.BIGINT()], result_type=DataTypes.BIGINT())
+def add(i, j):
+ import logging
+ import sys
+ logging.basicConfig(stream=sys.stdout, level=logging.INFO, format="%(message)s")
+ logging.info("debug")
+ return i + j
+
+{% endhighlight %}
+
+## Watching Logs
+
+If the environment variable `FLINK_HOME` is set, logs will be written in the log directory under `FLINK_HOME`.
+Otherwise, logs will be placed in the directory of the PyFlink module. You can execute the following command to find
+the log directory of PyFlink module:
+
+{% highlight shell %}
+
+$ python -c "import pyflink;import os;print(os.path.dirname(os.path.abspath(pyflink.__file__))+'/log')"
+
+{% endhighlight %}
+
+## Debug Python UDF
+You can take use of the pydevd_pycharm tool of PyCharm to debug Python UDF.
+
+1. Create a Python Remote Debug in PyCharm
+
+ run -> Python Remote Debug -> + -> choose a port (e.g. 6789)
+
+2. Install pydevd-pycharm tool
Review comment:
```suggestion
2. Install the `pydevd-pycharm` tool
```
##########
File path: docs/dev/python/user-guide/table/debugging.md
##########
@@ -0,0 +1,80 @@
+---
+title: "Debugging"
+nav-parent_id: python_tableapi
+nav-pos: 130
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+This page describes how to debug in PyFlink.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Logging Infos
+
+Python UDFs can log contextual and debug information via standard Python logging modules.
+
+{% highlight python %}
+
+@udf(input_types=[DataTypes.BIGINT(), DataTypes.BIGINT()], result_type=DataTypes.BIGINT())
+def add(i, j):
+ import logging
+ import sys
+ logging.basicConfig(stream=sys.stdout, level=logging.INFO, format="%(message)s")
+ logging.info("debug")
+ return i + j
+
+{% endhighlight %}
+
+## Watching Logs
Review comment:
```suggestion
## Accessing Logs
```
##########
File path: docs/dev/python/user-guide/table/debugging.md
##########
@@ -0,0 +1,80 @@
+---
+title: "Debugging"
+nav-parent_id: python_tableapi
+nav-pos: 130
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+This page describes how to debug in PyFlink.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Logging Infos
+
+Python UDFs can log contextual and debug information via standard Python logging modules.
+
+{% highlight python %}
+
+@udf(input_types=[DataTypes.BIGINT(), DataTypes.BIGINT()], result_type=DataTypes.BIGINT())
+def add(i, j):
+ import logging
+ import sys
+ logging.basicConfig(stream=sys.stdout, level=logging.INFO, format="%(message)s")
+ logging.info("debug")
+ return i + j
+
+{% endhighlight %}
+
+## Watching Logs
+
+If the environment variable `FLINK_HOME` is set, logs will be written in the log directory under `FLINK_HOME`.
+Otherwise, logs will be placed in the directory of the PyFlink module. You can execute the following command to find
+the log directory of PyFlink module:
+
+{% highlight shell %}
+
+$ python -c "import pyflink;import os;print(os.path.dirname(os.path.abspath(pyflink.__file__))+'/log')"
+
+{% endhighlight %}
+
+## Debug Python UDF
+You can take use of the pydevd_pycharm tool of PyCharm to debug Python UDF.
+
+1. Create a Python Remote Debug in PyCharm
+
+ run -> Python Remote Debug -> + -> choose a port (e.g. 6789)
+
+2. Install pydevd-pycharm tool
+
+ ```bash
+ $ pip install pydevd-pycharm
+ ```
+
+3. Add the following command in your Python UDF
+
+ ```python
+ import pydevd_pycharm
+ pydevd_pycharm.settrace('localhost', port=6789, stdoutToServer=True, stderrToServer=True)
+ ```
+
+4. Start the previous created Python Remote Debug Server
Review comment:
```suggestion
4. Start the previously created Python Remote Debug Server
```
##########
File path: docs/dev/python/user-guide/table/debugging.md
##########
@@ -0,0 +1,80 @@
+---
+title: "Debugging"
+nav-parent_id: python_tableapi
+nav-pos: 130
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+This page describes how to debug in PyFlink.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Logging Infos
+
+Python UDFs can log contextual and debug information via standard Python logging modules.
+
+{% highlight python %}
+
+@udf(input_types=[DataTypes.BIGINT(), DataTypes.BIGINT()], result_type=DataTypes.BIGINT())
+def add(i, j):
+ import logging
+ import sys
+ logging.basicConfig(stream=sys.stdout, level=logging.INFO, format="%(message)s")
+ logging.info("debug")
+ return i + j
+
+{% endhighlight %}
+
+## Watching Logs
+
+If the environment variable `FLINK_HOME` is set, logs will be written in the log directory under `FLINK_HOME`.
+Otherwise, logs will be placed in the directory of the PyFlink module. You can execute the following command to find
+the log directory of PyFlink module:
Review comment:
```suggestion
the log directory of the PyFlink module:
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org