[GitHub] [pulsar] liangyepianzhou commented on a diff in pull request #15239: [doc][tiered storage] read data from filesystem

[GitBox <gi...@apache.org>]

liangyepianzhou commented on code in PR #15239:
URL: https://github.com/apache/pulsar/pull/15239#discussion_r855938656


##########
site2/docs/tiered-storage-filesystem.md:
##########
@@ -520,4 +520,40 @@ Execute the following commands in the repository where you download Pulsar tarba
 
     And the **Capacity Used** is changed from 4 KB to 116.46 KB.
 
-    ![](assets/FileSystem-8.png)
\ No newline at end of file
+    ![](assets/FileSystem-8.png)
+
+## Read data from fileSystem
+
+This section provides detailed instructions about how to read data out as Ledger Entry in the file system.
+
+* The data was offloaded as `MapFile` to the following path:
+  ```properties
+    path = storageBasePath + "/" + managedLedgerName + "/" + ledgerId + "-" + uuid.toString();
+  ```
+    1. storageBasePath is the value of `hadoop.tmp.dir`, configured in `broker.conf` and `filesystem_offload_core_site.xml`
+    2. managedLedgerName is the name of the persistentTopic manager Ledger
+  ```shell
+     managedLedgerName of persistent://public/default/topics-name is  public/default/persistent/topics-name.
+  ```
+  Considering the iteration of versions,  you can use the following method to get the managedLedgerName:
+  ```shell
+     String managedLedgerName = TopicName.get("persistent://public/default/topics-name").getPersistenceNamingEncoding(); 
+  ```
+
+* Create a reader to read `MapFile` according to the above path and the `configuration` of the file system
+  ```shell
+     MapFile.Reader reader = new MapFile.Reader(new Path(dataFilePath),  configuration); 
+  ```
+* Read data as `LedgerEntry` from FileSystem.
+  ```java
+     LongWritable key = new LongWritable();
+     BytesWritable value = new BytesWritable();
+     key.set(nextExpectedId - 1);
+     reader.seek(key);
+     reader.next(key, value);
+     int length = value.getLength();
+     long entryId = key.get();
+     ByteBuf buf = PooledByteBufAllocator.DEFAULT.buffer(length, length);
+     buf.writeBytes(value.copyBytes());
+     LedgerEntryImpl ledgerEntry = LedgerEntryImpl.create(ledgerId, entryId, length, buf);

Review Comment:
   Great advice. I have added the deserialization steps, please review again when you have time.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org