[GitHub] [iceberg] aokolnychyi commented on pull request #5495: Spark 3.3: Reduce serialization in DeleteOrphanFilesSparkAction

[GitBox <gi...@apache.org>]

aokolnychyi commented on PR #5495:
URL: https://github.com/apache/iceberg/pull/5495#issuecomment-1218480126

   I played around with the benchmarks provided by @karuppayya and did not see a substantial impact on the time cause the rest of the functionality is so expensive and dominates the runtime. Also, I noticed that `UTF8String$toString` is quite efficient in our case as there is no data copy. On the other hand, `UTF8String.fromString` is more expensive. I did see extra conversions in the codegen before the change and also noticed the new approach has slightly less memory consumption. All in all, it is negligible on my laptop but I’d still go on with the change for the refactoring.
   
   That being said, we should compare the new implementation to the old solution we had in the previous release.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org