Re: [I] [C++] Add filesystem stats [arrow]

["wjones127 (via GitHub)" <gi...@apache.org>]

wjones127 commented on issue #38465:
URL: https://github.com/apache/arrow/issues/38465#issuecomment-1807003744

   FWIW in Lance what we've found most helpful in understanding filesystem / object store use is traces rather than metrics. We use the Rust `tracing` library and [Perfetto](https://perfetto.dev/) to visualize. Here's an example investigation: https://github.com/lancedb/lance/issues/1352#issuecomment-1753407265
   
   I think in Arrow C++, you could integrate OpenTelemetry tracing in the filesystems. @amoeba did some similar work integrating it with Flight C++.
   
   That being said, collecting these kinds of metrics would be useful for additional information in benchmark comparisons. It would be nice, for example, for a benchmark to show that a certain optimization reduced that number of IO requests.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org