[GitHub] [arrow] westonpace commented on a change in pull request #12609: ARROW-15067: [C++] Add tracing spans to the scanner

[GitBox <gi...@apache.org>]

westonpace commented on a change in pull request #12609:
URL: https://github.com/apache/arrow/pull/12609#discussion_r840921398



##########
File path: cpp/src/arrow/util/tracing_internal.h
##########
@@ -146,6 +158,17 @@ opentelemetry::trace::StartSpanOptions SpanOptionsWithParent(
         return st;                                                                \
       })
 
+#define PROPAGATE_SPAN_TO_GENERATOR(generator)                                \

Review comment:
       I accept your rationale for these macros but could we add a brief comment with that rationale above these one-liner macros?  Otherwise I'm afraid I'll end up asking you yet again :laughing: 

##########
File path: cpp/src/arrow/dataset/file_csv.cc
##########
@@ -167,9 +168,14 @@ static inline Result<csv::ReadOptions> GetReadOptions(
 static inline Future<std::shared_ptr<csv::StreamingReader>> OpenReaderAsync(
     const FileSource& source, const CsvFileFormat& format,
     const std::shared_ptr<ScanOptions>& scan_options, Executor* cpu_executor) {
+#ifdef ARROW_WITH_OPENTELEMETRY
+  auto tracer = arrow::internal::tracing::GetTracer();
+  auto span = tracer->StartSpan("arrow::dataset::CsvFileFormat::OpenReaderAsync");
+#endif
   ARROW_ASSIGN_OR_RAISE(auto reader_options, GetReadOptions(format, scan_options));
 
   ARROW_ASSIGN_OR_RAISE(auto input, source.OpenCompressed());
+  auto path = source.path();

Review comment:
       ```suggestion
     const auto& path = source.path();
   ```
   

##########
File path: cpp/src/arrow/util/tracing_internal.h
##########
@@ -101,6 +102,69 @@ AsyncGenerator<T> WrapAsyncGenerator(AsyncGenerator<T> wrapped,
   };
 }
 
+/// \brief Start a new span for each invocation of a generator.
+///
+/// The parent span of the new span will be the currently active span
+/// (if any) as of when WrapAsyncGenerator was itself called.
+template <typename T>
+AsyncGenerator<T> WrapAsyncGenerator(AsyncGenerator<T> wrapped,
+                                     const std::string& span_name) {
+  opentelemetry::trace::StartSpanOptions options;
+  options.parent = GetTracer()->GetCurrentSpan()->GetContext();
+  return WrapAsyncGenerator(std::move(wrapped), std::move(options), span_name);
+}
+
+/// \brief End the given span when the given async generator ends.
+///
+/// The span will be made the active span each time the generator is called.
+template <typename T>
+AsyncGenerator<T> TieSpanToAsyncGenerator(
+    AsyncGenerator<T> wrapped,
+    opentelemetry::nostd::shared_ptr<opentelemetry::trace::Span> span) {
+  return [=]() mutable -> Future<T> {
+    auto scope = GetTracer()->WithActiveSpan(span);
+    return wrapped().Then(
+        [span](const T& result) -> Result<T> {
+          span->SetStatus(opentelemetry::trace::StatusCode::kOk);
+          span->End();
+          return result;
+        },
+        [span](const Status& status) -> Result<T> {
+          MarkSpan(status, span.get());
+          span->End();
+          return status;
+        });
+  };
+}
+
+/// \brief End the given span when the given async generator ends.
+///
+/// The span will be made the active span each time the generator is called.
+template <typename T>
+AsyncGenerator<T> TieSpanToAsyncGenerator(AsyncGenerator<T> wrapped) {
+  auto span = GetTracer()->GetCurrentSpan();
+  return TieSpanToAsyncGenerator(wrapped, span);
+}
+
+/// \brief Activate the given span on each invocation of an async generator.
+template <typename T>
+AsyncGenerator<T> PropagateSpanThroughAsyncGenerator(
+    AsyncGenerator<T> wrapped,
+    opentelemetry::nostd::shared_ptr<opentelemetry::trace::Span> span) {
+  return [=]() mutable -> Future<T> {
+    auto scope = GetTracer()->WithActiveSpan(span);
+    return wrapped();
+  };
+}
+
+/// \brief Activate the given span on each invocation of an async generator.
+template <typename T>
+AsyncGenerator<T> PropagateSpanThroughAsyncGenerator(AsyncGenerator<T> wrapped) {
+  auto span = GetTracer()->GetCurrentSpan();
+  if (!span->GetContext().IsValid()) return wrapped;
+  return PropagateSpanThroughAsyncGenerator(std::move(wrapped), std::move(span));
+}
+

Review comment:
       Yes, looks much better, thank you.  So if I understand correctly:
   
   Wrap -> The generator "is the activity"
   Propagate -> The generator "is a part of a larger activity"




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org