Re: [PR] Add deferrable support to `InfluxDB3Operator` [airflow]

["subhramit (via GitHub)" <gi...@apache.org>]

subhramit commented on code in PR #71976:
URL: https://github.com/apache/airflow/pull/71976#discussion_r3875345255


##########
providers/influxdb/src/airflow/providers/influxdb/operators/influxdb3.py:
##########
@@ -40,6 +40,11 @@ class InfluxDB3Operator(BaseOperator):
 
     :param sql: The SQL query to be executed
     :param influxdb3_conn_id: Reference to :ref:`InfluxDB 3 connection id <howto/connection:influxdb3>`.
+    :param deferrable: Run the query from the triggerer instead of holding a worker slot for its
+        duration. Requires ``influxdb3-python>=0.12.0``. Note that InfluxDB 3 streams results over
+        Arrow Flight rather than exposing a job that can be polled, so the whole result set still flows back
+        through XCom -- deferring helps with long-running queries returning modest result sets
+        (aggregations, freshness probes), not with very large extracts.

Review Comment:
   Was just meaning to say that, since the InfluxDB 3 client doesn’t expose a poll-based async job model here (see PR desc for details), `deferrable=True` releases the worker slot while the query runs in the triggerer, but the full result still comes back through XCom when the task resumes. So the main benefit is for long-running queries with relatively small result sets, not for very large extracts
   
   Refined with better wording now



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org