You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iotdb.apache.org by ji...@apache.org on 2019/10/10 07:50:27 UTC
[incubator-iotdb] 01/01: add python client example in user guide
This is an automated email from the ASF dual-hosted git repository.
jiangtian pushed a commit to branch add_python_cli_in_guide
in repository https://gitbox.apache.org/repos/asf/incubator-iotdb.git
commit 037057eda4e1816d059601fd75342c6a73975c34
Author: jt <jt...@163.com>
AuthorDate: Thu Oct 10 15:38:24 2019 +0800
add python client example in user guide
---
client-py/readme.md | 28 ++--
docs/Documentation-CHN/OtherMaterial-Examples.md | 160 ++++++++++++++++++++++
docs/Documentation/OtherMaterial-Examples.md | 164 +++++++++++++++++++++++
3 files changed, 342 insertions(+), 10 deletions(-)
diff --git a/client-py/readme.md b/client-py/readme.md
index 8b79c94..9d849db 100644
--- a/client-py/readme.md
+++ b/client-py/readme.md
@@ -19,24 +19,32 @@
-->
-# introduction
-This is an example of how to connect to IoTDB with python, using the thrift rpc interfaces. Things will be a bit different
-on Linux or Windows, we will introduce how to operate on the two systems separately.
+# Python connection
+## introduction
+This is an example of how to connect to IoTDB with python, using the thrift rpc interfaces. Things
+are almost the same on Windows or Linux, but pay attention to the difference like path separator.
## Prerequisites
python3.7 or later is preferred.
You have to install Thrift (0.11.0 or later) to compile our thrift file into python code. Below is the official
-tutorial of installation:
+tutorial of installation, eventually, you should have a thrift executable.
```
http://thrift.apache.org/docs/install/
```
-## Compile
-If you have added Thrift executable into your path, you may just run `compile.sh` or `compile.bat`, or you will have to
-modify it to set variable `THRIFT_EXE` to point to your executable. This will generate thrift sources under folder `target`,
-you can add it to your `PYTHONPATH` so that you would be able to use the library in your code.
+## Compile the thrift library
+If you have added Thrift executable into your path, you may just run `client-py/compile.sh` or
+ `client-py/compile.bat`, or you will have to modify it to set variable `THRIFT_EXE` to point to
+your executable. This will generate thrift sources under folder `target`, you can add it to your
+`PYTHONPATH` so that you would be able to use the library in your code. Notice that the scripts
+locate the thrift source file by relative path, so if you move the scripts else where, they are
+no longer valid.
+
+Optionally, if you know the basic usage of thrift, you can only download the thrift source file in
+`service-rpc\src\main\thrift\rpc.thrift`, and simply use `thrift -gen py -out ./target rpc.thrift`
+to generate the python library.
## Example
-We provided an example of how to use the thrift library to connect to IoTDB in `src\client_example.py`, please read it
-carefully before you write your own code.
+We provided an example of how to use the thrift library to connect to IoTDB in `client-py/src
+/client_example.py`, please read it carefully before you write your own code.
\ No newline at end of file
diff --git a/docs/Documentation-CHN/OtherMaterial-Examples.md b/docs/Documentation-CHN/OtherMaterial-Examples.md
index 5ae6f4b..c6d6139 100644
--- a/docs/Documentation-CHN/OtherMaterial-Examples.md
+++ b/docs/Documentation-CHN/OtherMaterial-Examples.md
@@ -81,3 +81,163 @@ public class IotdbHelloWorld {
```
+# 使用Python进行连接
+## 介绍
+以下例子展示了如何使用Thrift编译出Python的库，并调用该库连接IoTDB并进行一些基本操作。在Windows和Linux上的
+使用大同小异，但请注意二者在路径等方面的细微区别。
+
+## Prerequisites
+建议使用python3.7或更高级的版本。
+
+您需要预先安装Thrift才能将我们的Thrift源文件编译为Python的库代码。下面是Thrift的官方安装教程，最终，您需要
+得到一个Thrift的二进制可执行文件。
+```
+http://thrift.apache.org/docs/install/
+```
+
+## Compile the thrift library
+如果您已经将Thrift的可执行文件放到了您的PATH环境变量的目录下，您可以直接运行脚本`client-py/compile.sh`或
+`client-py/compile.bat`，否则您需要修改脚本中的变量`THRIFT_EXE`来指向您的Thrift可执行文件。这会在脚本文件
+的同一目录下生成`target`文件夹，该文件夹即包含了IoTDB的Python版本的Thrift库，当您将该文件夹加入到您的`PYTHONPATH`
+环境变量里后，您就可以在自己的代码中引用该库。请注意，上述脚本通过相对路径寻找我们的Thrift的源文件，如果您改变
+脚本所在位置，脚本可能失效。
+
+或者，如果您对Thrift的基本用法有所了解，您就不需要下载整个项目，您可以仅下载我们的Thrift源文件`service-rpc\src\main\thrift\rpc.thrift`，
+并使用命令`thrift -gen py -out ./target rpc.thrift`来编译Thrift库。
+
+## Example
+下面是一段使用生成的Thrift库连接IoTDB的示例代码，请在编写您自己的代码之前仔细地阅读。
+```python
+import sys, struct
+sys.path.append("../target")
+
+from thrift.protocol import TBinaryProtocol
+from thrift.transport import TSocket, TTransport
+
+from rpc.TSIService import Client, TSCreateTimeseriesReq, TSInsertionReq, TSBatchInsertionReq, TSExecuteStatementReq,\
+ TS_SessionHandle, TSHandleIdentifier, TSOpenSessionReq, TSQueryDataSet, TSFetchResultsReq, TSCloseOperationReq,\
+ TSCloseSessionReq
+
+TSDataType = {
+ 'BOOLEAN' : 0,
+ 'INT32' : 1,
+ 'INT64' : 2,
+ 'FLOAT' : 3,
+ 'DOUBLE' : 4,
+ 'TEXT' : 5
+}
+
+TSEncoding = {
+ 'PLAIN' : 0,
+ 'PLAIN_DICTIONARY' : 1,
+ 'RLE' : 2,
+ 'DIFF' : 3,
+ 'TS_2DIFF' : 4,
+ 'BITMAP' : 5,
+ 'GORILLA' : 6,
+ 'REGULAR' : 7
+}
+
+Compressor = {
+ 'UNCOMPRESSED' : 0,
+ 'SNAPPY' : 1,
+ 'GZIP' : 2,
+ 'LZO' : 3,
+ 'SDT' : 4,
+ 'PAA' : 5,
+ 'PLA' : 6
+}
+
+
+if __name__ == '__main__':
+ ip = "localhost"
+ port = "6667"
+ username = 'root'
+ password = 'root'
+ # Make socket
+ transport = TSocket.TSocket(ip, port)
+
+ # Buffering is critical. Raw sockets are very slow
+ transport = TTransport.TBufferedTransport(transport)
+
+ # Wrap in a protocol
+ protocol = TBinaryProtocol.TBinaryProtocol(transport)
+
+ # Create a client to use the protocol encoder
+ client = Client(protocol)
+
+ # Connect!
+ transport.open()
+
+ # Authentication
+ client.openSession(TSOpenSessionReq(username=username, password=password))
+
+ # This is necessary for resource control
+ stmtId = client.requestStatementId()
+
+ # These two fields do not matter
+ handle = TS_SessionHandle(TSHandleIdentifier(b'uuid', b'secret'))
+
+ # create a storage group
+ status = client.setStorageGroup("root.group1")
+ print(status.statusType)
+
+ # create timeseries
+ status = client.createTimeseries(TSCreateTimeseriesReq("root.group1.s1", TSDataType['INT64'], TSEncoding['PLAIN'],
+ Compressor['UNCOMPRESSED']))
+ print(status.statusType)
+ status = client.createTimeseries(TSCreateTimeseriesReq("root.group1.s2", TSDataType['INT64'], TSEncoding['PLAIN'],
+ Compressor['UNCOMPRESSED']))
+ print(status.statusType)
+ status = client.createTimeseries(TSCreateTimeseriesReq("root.group1.s3", TSDataType['INT64'], TSEncoding['PLAIN'],
+ Compressor['UNCOMPRESSED']))
+ print(status.statusType)
+
+ # insert a single row
+ status = client.insertRow(TSInsertionReq("root.group1", ["s1", "s2", "s3"], ["1", "11", "111"], 1, 1))
+ print(status.statusType)
+
+ # insert multiple rows, this interface is more efficient
+ values = bytearray()
+ times = bytearray()
+ deviceId = "root.group1"
+ measurements = ["s1", "s2", "s3"]
+ dataSize = 3
+ dataTypes = [TSDataType['INT64'], TSDataType['INT64'], TSDataType['INT64']]
+ # the first 3 belong to 's1', the mid 3 belong to 's2', the last 3 belong to 's3'
+ values.extend(struct.pack('>qqqqqqqqq', 2, 3, 4, 22, 33, 44, 222, 333, 444))
+ times.extend(struct.pack('>qqq', 2, 3, 4))
+ resp = client.insertBatch(TSBatchInsertionReq(deviceId, measurements, values, times,
+ dataTypes, dataSize))
+ status = resp.status
+ print(status.statusType)
+
+ # execute deletion (or other statements)
+ resp = client.executeStatement(TSExecuteStatementReq(handle, "DELETE FROM root.group1 where time < 2"))
+ status = resp.status
+ print(status.statusType)
+
+ # query the data
+ stmt = "SELECT * FROM root.group1"
+ fetchSize = 2
+ # this is also for resource control, make sure different queries will not use the same id at the same time
+ queryId = 1
+ resp = client.executeQueryStatement(TSExecuteStatementReq(handle, stmt))
+ stmtHandle = resp.operationHandle
+ status = resp.status
+ print(status.statusType)
+ while True:
+ rst = client.fetchResults(TSFetchResultsReq(stmt, fetchSize, queryId)).queryDataSet
+ records = rst.records
+ if len(records) == 0:
+ break
+ for record in records:
+ print(record)
+
+ # do not forget to close it when a query is over
+ client.closeOperation(TSCloseOperationReq(stmtHandle, queryId, stmtId))
+
+ # and do not forget to close the session before exiting
+ client.closeSession(TSCloseSessionReq(handle))
+
+```
\ No newline at end of file
diff --git a/docs/Documentation/OtherMaterial-Examples.md b/docs/Documentation/OtherMaterial-Examples.md
index c56b650..069b25d 100644
--- a/docs/Documentation/OtherMaterial-Examples.md
+++ b/docs/Documentation/OtherMaterial-Examples.md
@@ -82,3 +82,167 @@ public class IotdbHelloWorld {
```
+
+# Python connection
+## introduction
+This is an example of how to connect to IoTDB with python, using the thrift rpc interfaces. Things
+are almost the same on Windows or Linux, but pay attention to the difference like path separator.
+
+## Prerequisites
+python3.7 or later is preferred.
+
+You have to install Thrift (0.11.0 or later) to compile our thrift file into python code. Below is the official
+tutorial of installation, eventually, you should have a thrift executable.
+```
+http://thrift.apache.org/docs/install/
+```
+
+## Compile the thrift library
+If you have added Thrift executable into your path, you may just run `client-py/compile.sh` or
+ `client-py/compile.bat`, or you will have to modify it to set variable `THRIFT_EXE` to point to
+your executable. This will generate thrift sources under folder `target`, you can add it to your
+`PYTHONPATH` so that you would be able to use the library in your code. Notice that the scripts
+locate the thrift source file by relative path, so if you move the scripts else where, they are
+no longer valid.
+
+Optionally, if you know the basic usage of thrift, you can only download the thrift source file in
+`service-rpc\src\main\thrift\rpc.thrift`, and simply use `thrift -gen py -out ./target rpc.thrift`
+to generate the python library.
+
+## Example
+We provided an example of how to use the thrift library to connect to IoTDB below, please read it
+carefully before you write your own code.
+```python
+import sys, struct
+sys.path.append("../target")
+
+from thrift.protocol import TBinaryProtocol
+from thrift.transport import TSocket, TTransport
+
+from rpc.TSIService import Client, TSCreateTimeseriesReq, TSInsertionReq, TSBatchInsertionReq, TSExecuteStatementReq,\
+ TS_SessionHandle, TSHandleIdentifier, TSOpenSessionReq, TSQueryDataSet, TSFetchResultsReq, TSCloseOperationReq,\
+ TSCloseSessionReq
+
+TSDataType = {
+ 'BOOLEAN' : 0,
+ 'INT32' : 1,
+ 'INT64' : 2,
+ 'FLOAT' : 3,
+ 'DOUBLE' : 4,
+ 'TEXT' : 5
+}
+
+TSEncoding = {
+ 'PLAIN' : 0,
+ 'PLAIN_DICTIONARY' : 1,
+ 'RLE' : 2,
+ 'DIFF' : 3,
+ 'TS_2DIFF' : 4,
+ 'BITMAP' : 5,
+ 'GORILLA' : 6,
+ 'REGULAR' : 7
+}
+
+Compressor = {
+ 'UNCOMPRESSED' : 0,
+ 'SNAPPY' : 1,
+ 'GZIP' : 2,
+ 'LZO' : 3,
+ 'SDT' : 4,
+ 'PAA' : 5,
+ 'PLA' : 6
+}
+
+
+if __name__ == '__main__':
+ ip = "localhost"
+ port = "6667"
+ username = 'root'
+ password = 'root'
+ # Make socket
+ transport = TSocket.TSocket(ip, port)
+
+ # Buffering is critical. Raw sockets are very slow
+ transport = TTransport.TBufferedTransport(transport)
+
+ # Wrap in a protocol
+ protocol = TBinaryProtocol.TBinaryProtocol(transport)
+
+ # Create a client to use the protocol encoder
+ client = Client(protocol)
+
+ # Connect!
+ transport.open()
+
+ # Authentication
+ client.openSession(TSOpenSessionReq(username=username, password=password))
+
+ # This is necessary for resource control
+ stmtId = client.requestStatementId()
+
+ # These two fields do not matter
+ handle = TS_SessionHandle(TSHandleIdentifier(b'uuid', b'secret'))
+
+ # create a storage group
+ status = client.setStorageGroup("root.group1")
+ print(status.statusType)
+
+ # create timeseries
+ status = client.createTimeseries(TSCreateTimeseriesReq("root.group1.s1", TSDataType['INT64'], TSEncoding['PLAIN'],
+ Compressor['UNCOMPRESSED']))
+ print(status.statusType)
+ status = client.createTimeseries(TSCreateTimeseriesReq("root.group1.s2", TSDataType['INT64'], TSEncoding['PLAIN'],
+ Compressor['UNCOMPRESSED']))
+ print(status.statusType)
+ status = client.createTimeseries(TSCreateTimeseriesReq("root.group1.s3", TSDataType['INT64'], TSEncoding['PLAIN'],
+ Compressor['UNCOMPRESSED']))
+ print(status.statusType)
+
+ # insert a single row
+ status = client.insertRow(TSInsertionReq("root.group1", ["s1", "s2", "s3"], ["1", "11", "111"], 1, 1))
+ print(status.statusType)
+
+ # insert multiple rows, this interface is more efficient
+ values = bytearray()
+ times = bytearray()
+ deviceId = "root.group1"
+ measurements = ["s1", "s2", "s3"]
+ dataSize = 3
+ dataTypes = [TSDataType['INT64'], TSDataType['INT64'], TSDataType['INT64']]
+ # the first 3 belong to 's1', the mid 3 belong to 's2', the last 3 belong to 's3'
+ values.extend(struct.pack('>qqqqqqqqq', 2, 3, 4, 22, 33, 44, 222, 333, 444))
+ times.extend(struct.pack('>qqq', 2, 3, 4))
+ resp = client.insertBatch(TSBatchInsertionReq(deviceId, measurements, values, times,
+ dataTypes, dataSize))
+ status = resp.status
+ print(status.statusType)
+
+ # execute deletion (or other statements)
+ resp = client.executeStatement(TSExecuteStatementReq(handle, "DELETE FROM root.group1 where time < 2"))
+ status = resp.status
+ print(status.statusType)
+
+ # query the data
+ stmt = "SELECT * FROM root.group1"
+ fetchSize = 2
+ # this is also for resource control, make sure different queries will not use the same id at the same time
+ queryId = 1
+ resp = client.executeQueryStatement(TSExecuteStatementReq(handle, stmt))
+ stmtHandle = resp.operationHandle
+ status = resp.status
+ print(status.statusType)
+ while True:
+ rst = client.fetchResults(TSFetchResultsReq(stmt, fetchSize, queryId)).queryDataSet
+ records = rst.records
+ if len(records) == 0:
+ break
+ for record in records:
+ print(record)
+
+ # do not forget to close it when a query is over
+ client.closeOperation(TSCloseOperationReq(stmtHandle, queryId, stmtId))
+
+ # and do not forget to close the session before exiting
+ client.closeSession(TSCloseSessionReq(handle))
+
+```
\ No newline at end of file