You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by "Cpaulyz (via GitHub)" <gi...@apache.org> on 2023/09/18 09:55:24 UTC

[GitHub] [iotdb] Cpaulyz opened a new pull request, #11170: [IOTDB-6162] Enhanced accuracy in deleting data

Cpaulyz opened a new pull request, #11170:
URL: https://github.com/apache/iotdb/pull/11170

   ## Description
   
   draft


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] Cpaulyz commented on pull request #11170: [IOTDB-6162] Enhanced accuracy in deleting data

Posted by "Cpaulyz (via GitHub)" <gi...@apache.org>.
Cpaulyz commented on PR #11170:
URL: https://github.com/apache/iotdb/pull/11170#issuecomment-1726783249

   Experiment environment:
   * mac M1, 1C1D, disable compaction
   * 1 DataRegion, 100 TimePartition, 1000 TsFile per time partition
   * execute `delete from root.sg1.d1.s1 where time<1209600000`, expect delete data in 1 time partitions
   
   
   Result:
   * before optimization
   <img width="1294" alt="image" src="https://github.com/apache/iotdb/assets/43774645/b8b2599b-db2c-4b91-82a7-096f6ad6ad8a">
   
   * after optimization
   <img width="1291" alt="image" src="https://github.com/apache/iotdb/assets/43774645/8cf8cf12-6e1f-431d-a926-e1811ceeaa6f">
   
   
   **There is a significant optimisation because when getting the TsfileResourceList, unwanted files can be filtered based on the start time and end time of the deletion, shortening the actual execution time of deleteDataInFiles.**
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] JackieTien97 merged pull request #11170: [IOTDB-6162] Enhanced accuracy in deleting data

Posted by "JackieTien97 (via GitHub)" <gi...@apache.org>.
JackieTien97 merged PR #11170:
URL: https://github.com/apache/iotdb/pull/11170


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] Cpaulyz commented on pull request #11170: [IOTDB-6162] Enhanced accuracy in deleting data

Posted by "Cpaulyz (via GitHub)" <gi...@apache.org>.
Cpaulyz commented on PR #11170:
URL: https://github.com/apache/iotdb/pull/11170#issuecomment-1726829353

   Experiment environment:
   * mac M1, 1C2D, disable compaction
   * 2 DataRegion, 10 time partion per region, 1000 TsFile per time partition
   * execute `delete from root.sg1.**.s1 where time>=11491200000`, expect delete file on 1 time partition locally
   <img width="898" alt="image" src="https://github.com/apache/iotdb/assets/43774645/feb3f8c6-9ac2-4fc1-bd82-6b4a811b3cf0">
   
   
   Result:
   * before 
   <img width="767" alt="image" src="https://github.com/apache/iotdb/assets/43774645/17ef9a4f-3e79-4a74-bba9-edfce15c5523">
   
   
   * after 
   <img width="1269" alt="image" src="https://github.com/apache/iotdb/assets/43774645/21db4bbc-9d92-4b69-9190-de78752c0513">
   
   
   **Experiments demonstrate that this optimisation leads to more accurate distributed plans being generated, thus improving execution efficiency when deleting data.**


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] Cpaulyz commented on pull request #11170: [IOTDB-6162] Enhanced accuracy in deleting data

Posted by "Cpaulyz (via GitHub)" <gi...@apache.org>.
Cpaulyz commented on PR #11170:
URL: https://github.com/apache/iotdb/pull/11170#issuecomment-1726830620

   Experiments demonstrate that this optimisation can make delete data about 5% to 10% more efficient in a set scenario. In more extreme scenarios, it may improve efficiency even more. PTAL~@JackieTien97 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org