[GitHub] [airflow] aa3pankaj commented on issue #12771: Instrument Airflow with Opentracing/Opentelemetry

[GitBox <gi...@apache.org>]

aa3pankaj commented on issue #12771:
URL: https://github.com/apache/airflow/issues/12771#issuecomment-1029932364


   @dm03514 thanks, i tried your test code, it is exporting spans successfully.
   But exporting spans directly from tasks would be much useful in terms of tracing different operations like hook execution, api calls etc.
   
   @mjpieters I tried exporting span from a airflow task:
   
   packages:
   apache-airflow==2.1.3
   opentelemetry-api==1.9.1
   opentelemetry-sdk==1.9.1
   opentelemetry-exporter-otlp==1.9.1
   opentelemetry-instrumentation==0.28b1
   
   env variables related to opentelemetry:
   OTEL_TRACES_EXPORTER=otlp
   OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:55680" OTEL_RESOURCE_ATTRIBUTES="service.name=test_airflow_worker"
   OTEL_TRACES_SAMPLER="always_on" 
   
   
   Airflow task:
   
   ```
   from airflow.models import BaseOperator
   from opentelemetry import trace
   from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
   from opentelemetry.sdk.resources import Resource
   from opentelemetry.sdk.trace import TracerProvider
   from opentelemetry.sdk.trace.export import BatchSpanProcessor
   
   
   class TracingTestOperator(BaseOperator): 
   
       def execute(self, context):
   
           resource = Resource(attributes={
               "service.name": "test_airflow_worker"
           })
   
           trace.set_tracer_provider(TracerProvider(resource=resource))
           tracer = trace.get_tracer(__name__)
   
           otlp_exporter = OTLPSpanExporter(endpoint="http://localhost:55680", insecure=True)
   
           span_processor = BatchSpanProcessor(otlp_exporter)
   
           trace.get_tracer_provider().add_span_processor(span_processor)
   
           with tracer.start_as_current_span("test_task_span"):
               print("Hello Airflow!")
   ```
   
   Above code is not exporting spans to the collector, even though collector (otlp) is up.
   Strangely, same code is exporting span successfully when I run it as standalone python (directly invoking execute method).
   
   
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org