You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2019/11/04 03:18:25 UTC
[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request
#513: [IOTDB-215]Update TsFile sketch tool and TsFile docs for v0.9.0
jt2594838 commented on a change in pull request #513: [IOTDB-215]Update TsFile sketch tool and TsFile docs for v0.9.0
URL: https://github.com/apache/incubator-iotdb/pull/513#discussion_r341894983
##########
File path: docs/Documentation/UserGuide/8-System Design (Developer)/1-Hierarchy.md
##########
@@ -296,92 +290,154 @@ If size of props is greater than 0, there is an array of <String, String> pair a
Such as "max_point_number""2".
-## Done
+##### 1.2.3.3 TsFileMetadataSize
+
+After the TsFileMetaData, there is an int indicating the size of the TsFileMetaData.
+
-After the `FileMetaData`, there will be another Magic String and you have finished the journey of discovering TsFile!
+#### 1.2.4 Magic String
+A TsFile ends with a 6-byte magic string (`TsFile`).
-You can also use /tsfile/example/TsFileSequenceRead to read and validate a TsFile.
-## TsFile Sketch Tool
-`org.apache.iotdb.tsfile.TsFileSketchTool` under the example/tsfile module is a tool to help you dive into the physical storage layout of a specific TsFile.
+Congratulations! You have finished the journey of discovering TsFile.
-Command:
-java -cp `<path of tsfile-example-0.9.0-SNAPSHOT.jar>`
-`<path of tsfile-0.9.0-SNAPSHOT-jar-with-dependencies.jar>`
-`org.apache.iotdb.tsfile.TsFileSketchTool`
-`<path of your TsFile>` (`<path of the file for saving the output result>`)
+### 1.3 Tool Set
-Note:
-- if `<path of the file for saving the output result>` is not set,
-the default path "TsFile_sketch_view.txt" will be used.
+#### 1.3.1 TsFileResource Print Tool
+
+After building the server, the startup script of this tool will appear under the `server\target\iotdb-server-0.9.0-SNAPSHOT\tools` directory.
+
+Command:
+
+For Windows:
-Command example:
```
-java -cp tsfile-example-0.9.0-SNAPSHOT.jar;D:\incubator-iotdb\tsfile\target\tsfile-0.9.0-SNAPSHOT-jar-with-dependencies.jar org.apache.iotdb.tsfile.TsFileSketchTool D:\1567769299010-101.tsfile
+.\print-tsfile-sketch.bat <path of your TsFileResource Directory>
```
-An example output result:
+For Linux or MacOs:
+
+```
+./print-tsfile-sketch.sh <path of your TsFileResource Directory>
+```
+
+An example on Windows:
+
+```
+D:\incubator-iotdb\server\target\iotdb-server-0.9.0-SNAPSHOT\tools>.\print-tsfile-resource-files.bat D:\data\data\sequence\root.vehicle
+````````````````````````
+Starting Printing the TsFileResources
+````````````````````````
+12:31:59.861 [main] WARN org.apache.iotdb.db.conf.IoTDBDescriptor - Cannot find IOTDB_HOME or IOTDB_CONF environment variable when loading config file iotdb-engine.properties, use default configuration
+analyzing D:\data\data\sequence\root.vehicle\1572496142067-101-0.tsfile ...
+device root.vehicle.d0, start time 3000 (1970-01-01T08:00:03+08:00[GMT+08:00]), end time 100999 (1970-01-01T08:01:40.999+08:00[GMT+08:00])
+analyzing the resource file finished.
+```
+
+#### 1.3.2 TsFile Sketch Tool
+
+After building the server, the startup script of this tool will appear under the `server\target\iotdb-server-0.9.0-SNAPSHOT\tools` directory.
+
+Command:
+
+For Windows:
+
+```
+.\print-tsfile-sketch.bat <path of your TsFile> (<path of the file for saving the output result>)
+```
+
+- Note that if `<path of the file for saving the output result>` is not set, the default path "TsFile_sketch_view.txt" will be used.
+
+For Linux or MacOs:
+
+```
+./print-tsfile-sketch.sh <path of your TsFile> (<path of the file for saving the output result>)
+```
+
+- Note that if `<path of the file for saving the output result>` is not set, the default path "TsFile_sketch_view.txt" will be used.
+
+An example on Windows:
+
```$xslt
+D:\incubator-iotdb\server\target\iotdb-server-0.9.0-SNAPSHOT\tools>.\print-tsfile-sketch.bat D:\data\data\sequence\root.vehicle\1572496142067-101-0.tsfile
+````````````````````````
+Starting Printing the TsFile Sketch
+````````````````````````
+TsFile path:D:\data\data\sequence\root.vehicle\1572496142067-101-0.tsfile
+Sketch save path:TsFile_sketch_view.txt
-------------------------------- TsFile Sketch --------------------------------
-file path: D:\1568815495311-101.tsfile
-file length: 1959
-
- POSITION| CONTENT
- -------- -------
- 0| [magic head] TsFilev0.8.0
->>>>>>>>>>>>>>>>>>>>> [Chunk Group] of root.vehicle.d1 begins at pos 12, ends at pos 164, version:0, num of Chunks:1
- 12| [Chunk] of s0, numOfPoints:2, time range:[1,1000], tsDataType:INT32,
- TsDigest:[min_value:888,max_value:999,first_value:999,last_value:888,sum_value:1887.0]
- | [marker] 1
- | [ChunkHeader]
- | num of pages: 1
- | [marker] 0
- | [Chunk Group Footer]
-<<<<<<<<<<<<<<<<<<<<< [Chunk Group] of root.vehicle.d1 ends
->>>>>>>>>>>>>>>>>>>>> [Chunk Group] of root.vehicle.d0 begins at pos 164, ends at pos 989, version:0, num of Chunks:5
- 164| [Chunk] of s3, numOfPoints:6, time range:[60,946684800000], tsDataType:TEXT,
- TsDigest:[min_value:aaaaa,max_value:good,first_value:aaaaa,last_value:good,sum_value:0.0]
- | [marker] 1
- | [ChunkHeader]
- | num of pages: 1
- 366| [Chunk] of s4, numOfPoints:1, time range:[100,100], tsDataType:BOOLEAN,
- TsDigest:[min_value:true,max_value:true,first_value:true,last_value:true,sum_value:0.0]
- | [marker] 1
- | [ChunkHeader]
- | num of pages: 1
- 461| [Chunk] of s0, numOfPoints:11, time range:[1,1000], tsDataType:INT32,
- TsDigest:[min_value:80,max_value:22222,first_value:101,last_value:22222,sum_value:42988.0]
- | [marker] 1
- | [ChunkHeader]
- | num of pages: 1
- 614| [Chunk] of s1, numOfPoints:11, time range:[1,946684800000], tsDataType:INT64,
- TsDigest:[min_value:100,max_value:55555,first_value:1101,last_value:100,sum_value:147922.0]
- | [marker] 1
- | [ChunkHeader]
- | num of pages: 1
- 822| [Chunk] of s2, numOfPoints:6, time range:[2,1000], tsDataType:FLOAT,
- TsDigest:[min_value:2.22,max_value:1000.11,first_value:2.22,last_value:1000.11,sum_value:1031.2099850177765]
- | [marker] 1
- | [ChunkHeader]
- | num of pages: 1
- | [marker] 0
- | [Chunk Group Footer]
-<<<<<<<<<<<<<<<<<<<<< [Chunk Group] of root.vehicle.d0 ends
- 989| [marker] 2
- 990| [TsDeviceMetadata] of root.vehicle.d0, startTime:1, endTime:946684800000
- | num of ChunkGroupMetaData: 1
- 1553| [TsDeviceMetadata] of root.vehicle.d1, startTime:1, endTime:1000
- | num of ChunkGroupMetaData: 1
- 1718| [TsFileMetaData]
- | num of TsDeviceMetadataIndex: 2
- | num of measurementSchema: 5
- 1943| [TsFileMetaDataSize] 225
- 1947| [magic tail] TsFilev0.8.0
- 1959| END of TsFile
+file path: D:\data\data\sequence\root.vehicle\1572496142067-101-0.tsfile
+file length: 187382
+
+ POSITION| CONTENT
+ -------- -------
+ 0| [magic head] TsFile
+ 6| [version number] 000001
+||||||||||||||||||||| [Chunk Group] of root.vehicle.d0 begins at pos 12, ends at pos 186469, version:102, num of Chunks:6
+ 12| [Chunk] of s3, numOfPoints:10600, time range:[3000,13599], tsDataType:TEXT,
+ TsDigest:[min_value:A,max_value:E,first_value:A,last_value:E,sum_value:0.0]
+ | [marker] 1
+ | [ChunkHeader]
+ | 11 pages
+ 55718| [Chunk] of s4, numOfPoints:10600, time range:[3000,13599], tsDataType:BOOLEAN,
+ TsDigest:[min_value:false,max_value:true,first_value:true,last_value:false,sum_value:0.0]
+ | [marker] 1
+ | [ChunkHeader]
+ | 11 pages
+ 68848| [Chunk] of s5, numOfPoints:10600, time range:[3000,13599], tsDataType:DOUBLE,
+ TsDigest:[min_value:3000.0,max_value:13599.0,first_value:3000.0,last_value:13599.0,sum_value:8.79747E7]
+ | [marker] 1
+ | [ChunkHeader]
+ | 11 pages
+ 98474| [Chunk] of s0, numOfPoints:21900, time range:[3000,100999], tsDataType:INT32,
+ TsDigest:[min_value:0,max_value:99,first_value:0,last_value:19,sum_value:889750.0]
+ | [marker] 1
+ | [ChunkHeader]
+ | 22 pages
+ 123369| [Chunk] of s1, numOfPoints:21900, time range:[3000,100999], tsDataType:INT64,
+ TsDigest:[min_value:0,max_value:39,first_value:8,last_value:19,sum_value:300386.0]
+ | [marker] 1
+ | [ChunkHeader]
+ | 22 pages
+ 144741| [Chunk] of s2, numOfPoints:21900, time range:[3000,100999], tsDataType:FLOAT,
+ TsDigest:[min_value:0.0,max_value:122.0,first_value:8.0,last_value:52.0,sum_value:778581.0]
+ | [marker] 1
+ | [ChunkHeader]
+ | 22 pages
+ 186437| [Chunk Group Footer]
+ | [marker] 0
+ | [deviceID] root.vehicle.d0
+ | [dataSize] 186425
+ | [num of chunks] 6
+||||||||||||||||||||| [Chunk Group] of root.vehicle.d0 ends
+ 186469| [marker] 2
+ 186470| [TsDeviceMetadata] of root.vehicle.d0, startTime:3000, endTime:100999
+ | [startTime] 3000tfi
+ | [endTime] 100999
+ | [num of ChunkGroupMetaData] 1
+ | 1 ChunkGroupMetaData
+ 187133| [TsFileMetaData]
+ | [num of devices] 1
+ | 1 key&TsDeviceMetadataIndex
+ | [num of measurements] 6
+ | 6 key&measurementSchema
+ | [createBy isNotNull] false
+ | [totalChunkNum] 6
+ | [invalidChunkNum] 0
+ | [bloom filter bit vector byte array length] 31
+ | [bloom filter bit vector byte array]
+ | [bloom filter number of bits] 256
+ | [bloom filter number of hash functions] 5
+ 187372| [TsFileMetaDataSize] 239
+ 187376| [magic tail] TsFile
+ 187382| END of TsFile
---------------------------------- TsFile Sketch End ----------------------------------
```
-## A TsFile Visualization Example (v0.8.0)
-
+#### 1.3.3 TsFileSequenceRead
+You can also use `example/tsfile/org/apache/iotdb/tsfile/TsFileSequenceRead` to sequentially print a TsFile's content.
+
+### 1.4 A TsFile Visualization Example (v0.8.0)
+
Review comment:
Should this v0.8.0 be updated?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services