[GitHub] [arrow] jonkeane commented on pull request #11277: ARROW-14020: [R] Writing datafames with list columns is slow and scales poorly with nesting level

[GitBox <gi...@apache.org>]

jonkeane commented on pull request #11277:
URL: https://github.com/apache/arrow/pull/11277#issuecomment-933783636


   I benchmarked this a while ago (and none of the operative code has changed since then):
   
   > [Finished ⬇️0.67% ⬆️0.67%] ursa-i9-9960x
   
   The one regression is a false positive (in fact, it's in a Python benchmark!). [The simple features benchmark has improved 99%](https://conbench.ursa.dev/compare/benchmarks/acfa1ae5902d4a17ba124962cb252131...6540baeb176d426d894a73f41e9e428e/) this is actually cheating a bit because before it would serialize a bunch of metadata and now it skips it totally. We don't otherwise have a benchmark that touches this code (we could write one that includes a list column with no metadata, but I'm not certain we need that at this point) — I did test this manually with the code in the Jira and this case is, indeed, much faster.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org