[arrow-datafusion] branch main updated: Improve file scan time opening metric to include start_next_file (#5900)

[al...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

alamb pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow-datafusion.git


The following commit(s) were added to refs/heads/main by this push:
     new f8e2cc04c2 Improve file scan time opening metric to include start_next_file (#5900)
f8e2cc04c2 is described below

commit f8e2cc04c2b7fd281d748fc87fdc0b382263166a
Author: Andrew Lamb <an...@nerdnetworks.org>
AuthorDate: Mon Apr 10 11:02:27 2023 -0400

    Improve file scan time opening metric to include start_next_file (#5900)
---
 datafusion/core/src/physical_plan/file_format/file_stream.rs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/datafusion/core/src/physical_plan/file_format/file_stream.rs b/datafusion/core/src/physical_plan/file_format/file_stream.rs
index bb00170535..7890223457 100644
--- a/datafusion/core/src/physical_plan/file_format/file_stream.rs
+++ b/datafusion/core/src/physical_plan/file_format/file_stream.rs
@@ -264,9 +264,9 @@ impl<F: FileOpener> FileStream<F> {
                     Ok(reader) => {
                         let partition_values = mem::take(partition_values);
 
-                        let next = self.start_next_file().transpose();
-
+                        // include time needed to start opening in `start_next_file`
                         self.file_stream_metrics.time_opening.stop();
+                        let next = self.start_next_file().transpose();
                         self.file_stream_metrics.time_scanning_until_data.start();
                         self.file_stream_metrics.time_scanning_total.start();