You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Ajay9704 (via GitHub)" <gi...@apache.org> on 2026/01/07 09:06:39 UTC

[PR] Docs: clarify BaseSensorOperator params [airflow]

Ajay9704 opened a new pull request, #60203:
URL: https://github.com/apache/airflow/pull/60203

   ## Summary
   This PR resolves issue #60185 where BaseSensorOperator wasn't clickable in provider documentation, making it hard to find information about sensor parameters.
   
   ## The Problem
   I discovered that BaseSensorOperator lives in the new Task SDK at airflow/task-sdk/src/airflow/sdk/bases/sensor.py, but when building documentation for providers, the system only scanned the current provider's source code. This meant that when providers like the standard sensors referenced BaseSensorOperator, the documentation couldn't create proper links to the actual class definition.
   
   ## My Approach
   I modified airflow/devel-common/src/docs/provider_conf.py to:
   - Add the task-sdk source path to the autoapi directories so provider docs can access SDK classes
   - Set up proper intersphinx mapping to link to task-sdk documentation
   - Include safeguards to prevent any conflicts between documentation sources
   
   ## Testing Done
   I verified that:
   - Documentation builds still work properly with these changes
   - The links now work correctly in provider documentation
   - No existing functionality is broken
   - The fix is isolated to just the documentation generation process
   
   ## Impact
   This change means users can now click on BaseSensorOperator references in provider documentation and jump directly to the class definition to see all available parameters and methods. Much more user-friendly


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "Ajay9704 (via GitHub)" <gi...@apache.org>.
Ajay9704 commented on code in PR #60203:
URL: https://github.com/apache/airflow/pull/60203#discussion_r2667935778


##########
devel-common/src/docs/provider_conf.py:
##########
@@ -290,6 +298,19 @@
 # your API documentation from.
 autoapi_dirs = [BASE_PROVIDER_SRC_PATH.as_posix()]
 
+# Include task-sdk source path for proper linking of SDK classes like BaseSensorOperator
+TASK_SDK_PATH = AIRFLOW_REPO_ROOT_PATH / "task-sdk" / "src"
+if TASK_SDK_PATH.exists():
+    autoapi_dirs.append(TASK_SDK_PATH.as_posix())
+
+# Also update autoapi_ignore to prevent conflicts between provider and task-sdk docs
+# Ensure we don't exclude important SDK files that providers depend on
+TASK_SDK_PATH_STR = TASK_SDK_PATH.as_posix()
+if TASK_SDK_PATH_STR in autoapi_dirs:
+    # Adjust autoapi_ignore to make sure we don't exclude important SDK classes
+    # Remove any conflicting ignore patterns if needed
+    pass

Review Comment:
   @uranusjr , sir  I’ve removed it and cleaned up the code accordingly. Thanks for pointing it out and helping me ...
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "Ajay9704 (via GitHub)" <gi...@apache.org>.
Ajay9704 commented on PR #60203:
URL: https://github.com/apache/airflow/pull/60203#issuecomment-3722071676

   @potiuk  Sir , sorry for troubling you , but unfortunately my changes didn't worked for static checks , 
   this is my code
   -intersphinx_mapping["task-sdk"] = (f"[https://airflow.apache.org/docs/task-sdk/{task_sdk_version}/](https://airflow.apache.org/docs/task-sdk/%7Btask_sdk_version%7D/)", (f"[https://airflow.apache.org/docs/task-sdk/{task_sdk_version}/objects.inv](https://airflow.apache.org/docs/task-sdk/%7Btask_sdk_version%7D/objects.inv)",))
   ..........................................................................
   and pre-commit hooks chnaged this to 
   ..............................................................................
    intersphinx_mapping["task-sdk"] = (
      f"[https://airflow.apache.org/docs/task-sdk/{task_sdk_version}/](https://airflow.apache.org/docs/task-sdk/%7Btask_sdk_version%7D/)",
       (f"[https://airflow.apache.org/docs/task-sdk/{task_sdk_version}/objects.inv](https://airflow.apache.org/docs/task-sdk/%7Btask_sdk_version%7D/objects.inv)",),
   )
   I was unaware about this before and i apologise for the mistake and now , i have changed that part so now it should also pass static checks , so please if you don't mind re-approve to run all checks again , and if it passes then i can get the screenshots from the docs generated that you asked for from there , please do me this help sir


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on PR #60203:
URL: https://github.com/apache/airflow/pull/60203#issuecomment-3950320015

   please take a look at the CI failure. otherwise, I think we're almost good


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #60203:
URL: https://github.com/apache/airflow/pull/60203#discussion_r2667831645


##########
devel-common/src/docs/provider_conf.py:
##########
@@ -290,6 +298,19 @@
 # your API documentation from.
 autoapi_dirs = [BASE_PROVIDER_SRC_PATH.as_posix()]
 
+# Include task-sdk source path for proper linking of SDK classes like BaseSensorOperator
+TASK_SDK_PATH = AIRFLOW_REPO_ROOT_PATH / "task-sdk" / "src"
+if TASK_SDK_PATH.exists():
+    autoapi_dirs.append(TASK_SDK_PATH.as_posix())
+
+# Also update autoapi_ignore to prevent conflicts between provider and task-sdk docs
+# Ensure we don't exclude important SDK files that providers depend on
+TASK_SDK_PATH_STR = TASK_SDK_PATH.as_posix()
+if TASK_SDK_PATH_STR in autoapi_dirs:
+    # Adjust autoapi_ignore to make sure we don't exclude important SDK classes
+    # Remove any conflicting ignore patterns if needed
+    pass

Review Comment:
   This doesn’t do anything?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "Ajay9704 (via GitHub)" <gi...@apache.org>.
Ajay9704 commented on PR #60203:
URL: https://github.com/apache/airflow/pull/60203#issuecomment-3722919809

   Hi @potiuk  Sir,
   I’ve addressed all the issues that were actionable from my side and updated the code accordingly.
   Regarding the screenshots you asked for, i am currently unable to generate them locally because the documentation build hasn’t completed successfully yet ,from what all i can see, there doesn’t seem to be anything further I can fix on my end to unblock the docs build.
   Could you please advise on how I should proceed here?   Is there an alternative way to verify or demonstrate the clickable links, or anything else that you like me to try?
   
   @potiuk , @phanikumv  thank you very much for your patience and for helping by re-triggering the checks so far 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #60203:
URL: https://github.com/apache/airflow/pull/60203#discussion_r2844277015


##########
devel-common/src/docs/provider_conf.py:
##########
@@ -276,6 +275,24 @@
 if PACKAGE_NAME in ["apache-airflow-providers-google"]:
     intersphinx_mapping.update(get_google_intersphinx_mapping())
 
+# Add task-sdk to intersphinx mapping for proper cross-referencing of SDK classes
+# This allows proper linking to BaseSensorOperator and other SDK classes from provider docs
+try:
+    # Import airflow.sdk at runtime to get version for intersphinx mapping
+    # We need to import it here rather than at the top because the SDK may not be installed
+    # when building provider docs separately
+    import airflow.sdk as _airflow_sdk
+
+    task_sdk_version = parse_version(_airflow_sdk.__version__).base_version
+    intersphinx_mapping["task-sdk"] = (
+        f"https://airflow.apache.org/docs/task-sdk/{task_sdk_version}/",
+        (f"https://airflow.apache.org/docs/task-sdk/{task_sdk_version}/objects.inv",),
+    )
+except Exception:

Review Comment:
   ```suggestion
   except ModuleNotFoundError:
   ```



##########
airflow-core/docs/core-concepts/sensors.rst:
##########
@@ -32,89 +32,62 @@ Much like Operators, Airflow has a large set of pre-built Sensors you can use, b
 
 .. seealso:: :doc:`../authoring-and-scheduling/deferring`
 
-BaseSensorOperator parameters
------------------------------
+BaseSensorOperator
+-------------------
 
-All sensors in Airflow ultimately inherit from ``BaseSensorOperator`` (directly or indirectly).
-This base class defines the common behavior and parameters that control
-how a sensor waits, retries, and manages worker resources.
+All sensors share common functionality through the ``BaseSensorOperator`` class. This base class provides
+standardized parameters that control timing behavior, execution modes, and failure handling across
+every sensor implementation.
 
-As of the Task SDK refactor, ``BaseSensorOperator`` is implemented in the
-Task SDK. Because provider documentation is generated separately, these
-parameters may not always be directly visible on individual provider
-sensor API pages. However, they apply to *all* sensors.
+.. note::
+   In the Task SDK architecture, ``BaseSensorOperator`` is defined in the ``airflow.sdk`` package.
+   Provider documentation is generated separately from the core documentation, so sensor parameters
+   may not always appear in individual provider sensor API reference pages. However, these parameters
+   are universally available to all sensors.
 
-Common parameters
-^^^^^^^^^^^^^^^^^
+Common Sensor Parameters
+~~~~~~~~~~~~~~~~~~~~~~~~~
 
-The following parameters are provided by ``BaseSensorOperator`` and are
-available on all sensors:
+The following parameters control sensor behavior across all implementations:
 
 ``poke_interval``
-    Time in seconds between successive checks. In ``poke`` mode, the sensor
-    sleeps between checks while occupying a worker slot. In ``reschedule``
-    mode, the task is deferred and rescheduled after this interval.
+    Specifies how long to wait (in seconds) between successive checks. In ``poke`` mode, the sensor
+    holds a worker slot while waiting. In ``reschedule`` mode, the task is freed and rescheduled after
+    each interval. Accepts either a ``float`` (seconds) or ``timedelta`` object.
 
 ``timeout``
-    Maximum time in seconds the sensor is allowed to run before failing.
-    This timeout is measured from the first execution attempt, not per poke.
+    Maximum time (in seconds) before the sensor fails due to exceeding its allowed runtime.
+    The timeout is calculated from the first execution attempt, not from individual poke attempts.
+    This differs from ``execution_timeout`` which only measures active task execution time.
+    Accepts either a ``float`` (seconds) or ``timedelta`` object.
 
 ``mode``
-    Determines how the sensor occupies worker resources.
+    Controls how the sensor consumes worker resources:
 
-    * ``poke`` (default): occupies a worker slot for the entire duration
-    * ``reschedule``: releases the worker slot between checks
+    * ``poke`` (default): The sensor occupies a worker slot continuously while waiting
+    * ``reschedule``: The sensor releases the worker slot between checks, freeing resources for other tasks
 
 ``soft_fail``
-    If set to ``True``, the sensor will be marked as ``SKIPPED`` instead of
-    ``FAILED`` when the timeout is reached.
+    When ``True``, timeout causes the sensor to be marked as ``SKIPPED`` instead of ``FAILED``.
+    This is useful when you want non-critical sensors to timeout gracefully without failing the DAG.

Review Comment:
   ```suggestion
       This is useful when you want non-critical sensors to timeout gracefully without failing the Dag.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed pull request #60203: Docs: clarify BaseSensorOperator params
URL: https://github.com/apache/airflow/pull/60203


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "Ajay9704 (via GitHub)" <gi...@apache.org>.
Ajay9704 commented on PR #60203:
URL: https://github.com/apache/airflow/pull/60203#issuecomment-3718245776

   @uranusjr @phanikumv Thanks for the reviews! I've addressed the feedback:
   
   1. Removed the unnecessary try-except block around the airflow.sdk import since task-sdk must be available when building docs for providers that use BaseSensorOperator
   2. Removed the empty code block with just a pass statement that wasn't doing anything
   
   The changes now cleanly add:
    Task-sdk source path to autoapi_dirs so provider documentation can find SDK classes
    Intersphinx mapping to enable proper cross-referencing to task-sdk documentation
   
   This should resolve the issue where BaseSensorOperator wasn't clickable in provider documentation, making sensor parameters easier to discover. Thanks again for the feedback!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #60203:
URL: https://github.com/apache/airflow/pull/60203#issuecomment-3718773397

   Can you please show docs generated by it with clickable classes ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "Ajay9704 (via GitHub)" <gi...@apache.org>.
Ajay9704 commented on code in PR #60203:
URL: https://github.com/apache/airflow/pull/60203#discussion_r2667773483


##########
devel-common/src/docs/provider_conf.py:
##########
@@ -276,6 +276,18 @@
 if PACKAGE_NAME in ["apache-airflow-providers-google"]:
     intersphinx_mapping.update(get_google_intersphinx_mapping())
 
+# Add task-sdk to intersphinx mapping for proper cross-referencing of SDK classes
+# This allows proper linking to BaseSensorOperator and other SDK classes from provider docs
+try:
+    import airflow.sdk
+    # Add remote task-sdk inventory for cross-referencing
+    # This enables proper linking to SDK classes like BaseSensorOperator
+    task_sdk_version = parse_version(airflow.sdk.__version__).base_version
+    intersphinx_mapping["task-sdk"] = (f"https://airflow.apache.org/docs/task-sdk/{task_sdk_version}/", None)
+except ImportError:
+    # If task-sdk is not available, don't add the mapping
+    pass

Review Comment:
   @uranusjr You're right Sir , thanks for pointing this out. On reflection, I realize the try-except was overly defensive for this specific case. If we're building documentation for providers that use BaseSensorOperator, then task-sdk must be available since that's where BaseSensorOperator is defined.
   
   I was initially thinking about defensive programming for various build environments (CI, local dev, etc.), but you're correct that if someone's building docs for providers that inherit from BaseSensorOperator, the task-sdk has to be available. 
   
   I'll update the PR to remove the try-except block and just directly add the intersphinx mapping. Appreciate the feedback - definitely helps make the code cleaner and more straightforward!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "phanikumv (via GitHub)" <gi...@apache.org>.
phanikumv commented on code in PR #60203:
URL: https://github.com/apache/airflow/pull/60203#discussion_r2667813790


##########
devel-common/src/docs/provider_conf.py:
##########
@@ -321,6 +346,8 @@
 
     autoapi_ignore.extend(f"{d}/*" for d in test_dir.glob("*") if d.is_dir() and d.name != "system")
 
+
+

Review Comment:
   why this diff is introduced?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #60203:
URL: https://github.com/apache/airflow/pull/60203#issuecomment-3718896558

   Some mypy issue to fix @Ajay9704 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "Ajay9704 (via GitHub)" <gi...@apache.org>.
Ajay9704 commented on PR #60203:
URL: https://github.com/apache/airflow/pull/60203#issuecomment-3946794011

   
   
   Uploading Screencast From 2026-02-23 22-14-37.mp4…
   
   
   
   <img width="1920" height="1080" alt="Screenshot From 2026-02-24 00-18-26" src="https://github.com/user-attachments/assets/96afd051-e8ac-453d-b7f8-22d478125bd8" />
   <img width="1920" height="1080" alt="Screenshot From 2026-02-24 00-18-57" src="https://github.com/user-attachments/assets/fb956fd5-6b79-4fed-9b71-5ecd9225c7cc" />
   Please review and let me know if anything is still pending from my side.
   I’ve attached the screenshots and the screen recording of the fix for your reference.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #60203:
URL: https://github.com/apache/airflow/pull/60203#issuecomment-4194949464

   This draft pull request has had no activity from the author for over 3 weeks.
   
   **@Ajay9704**, you are welcome to reopen this PR when you are ready to continue working on it. Thank you for your contribution!
   
   <!-- Pull Request quality criteria -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #60203:
URL: https://github.com/apache/airflow/pull/60203#discussion_r2667664641


##########
devel-common/src/docs/provider_conf.py:
##########
@@ -276,6 +276,18 @@
 if PACKAGE_NAME in ["apache-airflow-providers-google"]:
     intersphinx_mapping.update(get_google_intersphinx_mapping())
 
+# Add task-sdk to intersphinx mapping for proper cross-referencing of SDK classes
+# This allows proper linking to BaseSensorOperator and other SDK classes from provider docs
+try:
+    import airflow.sdk
+    # Add remote task-sdk inventory for cross-referencing
+    # This enables proper linking to SDK classes like BaseSensorOperator
+    task_sdk_version = parse_version(airflow.sdk.__version__).base_version
+    intersphinx_mapping["task-sdk"] = (f"https://airflow.apache.org/docs/task-sdk/{task_sdk_version}/", None)
+except ImportError:
+    # If task-sdk is not available, don't add the mapping
+    pass

Review Comment:
   When is this ever not available…?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #60203:
URL: https://github.com/apache/airflow/pull/60203#issuecomment-3718777031

   I also rebased it and added "full tests needed" just in case.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "Ajay9704 (via GitHub)" <gi...@apache.org>.
Ajay9704 commented on PR #60203:
URL: https://github.com/apache/airflow/pull/60203#issuecomment-3827541753

   @phanikumv , @uranusjr please review my pr , I know the update turned out a bit long, but I’ve thoroughly explored all the issues that led to the checks failing and addressed each one.
   
   Please let me know if there’s anything else needed from my side.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #60203:
URL: https://github.com/apache/airflow/pull/60203#issuecomment-3726251372

   > Could you please advise on how I should proceed here? Is there an alternative way to verify or demonstrate the clickable links, or anything else that you like me to try?
   
   hard to say - but this is part of the task to figure out how to make the build succeed, I suggest you try to run it locally - following contributing guide, build it for one provider, look at the errors and try to fix it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "Ajay9704 (via GitHub)" <gi...@apache.org>.
Ajay9704 commented on code in PR #60203:
URL: https://github.com/apache/airflow/pull/60203#discussion_r2667929867


##########
devel-common/src/docs/provider_conf.py:
##########
@@ -321,6 +346,8 @@
 
     autoapi_ignore.extend(f"{d}/*" for d in test_dir.glob("*") if d.is_dir() and d.name != "system")
 
+
+

Review Comment:
    @phanikumv, I’ve removed it and cleaned up the code accordingly. Appreciate the feedback!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "Ajay9704 (via GitHub)" <gi...@apache.org>.
Ajay9704 commented on PR #60203:
URL: https://github.com/apache/airflow/pull/60203#issuecomment-3828013054

   @shahar1 @phanikumv , @uranusjr please review my pr , I know the update turned out a bit long, but I’ve thoroughly explored all the issues that led to the checks failing and addressed each one.
   
   Please let me know if there’s anything else needed from my side.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "Ajay9704 (via GitHub)" <gi...@apache.org>.
Ajay9704 commented on PR #60203:
URL: https://github.com/apache/airflow/pull/60203#issuecomment-3957636172

   I wanted to share a clear summary of everything I’ve tried so far regarding the BaseSensorOperator cross-reference issue. I have been working on this continuously for the past two days, and I even used AI at a few points just to understand parts of the structure better and debug the issue. Still, I haven’t been able to get to the root cause. At this point, I’m stuck and need some guidance from maintainers or reviewers.
   
   Below is everything I have tested from my side.
   
   1. Including task-sdk in autoapi_dirs
   Added the task-sdk modules directly into autoapi_dirs.
   This produced 140+ RST formatting errors due to non-Sphinx-compatible docstrings in modules like api/, datamodels/, execution_time/, etc.
   Result: Documentation build failed.
   
   2. Selective task-sdk inclusion using autoapi_ignore
   Tried to include only bases/sensor.py and ignore everything else.
   autoapi_ignore patterns didn’t work as expected and other task-sdk modules were still processed.
   Result: Same RST formatting errors as before.
   
   3. Using intersphinx mapping only
   Tried linking to task-sdk via intersphinx after building task-sdk separately.
   Provider docs don’t build task-sdk locally, so no inventory was available.
   Result: BaseSensorOperator still appeared as plain text.
   
   4. Enabling autoapi_generate_api_docs = True inside task-sdk
   Enabled full AutoAPI generation for task-sdk.
   This exposed multiple BaseOperator definitions across different files, which caused cross-reference conflicts.
   Result: Build failed with “more than one target found” errors.
   
   5. Adding intersphinx mapping for common-compat
   Added intersphinx mapping for apache-airflow-providers-common-compat.
   Provider docs still couldn’t resolve references to its inventory.
   Result: Link still not generated.
   
   6. Creating a manual BaseSensorOperator.rst file
   Created a proxy RST file pointing to airflow.sdk.bases.sensor.
   Sphinx still couldn’t resolve the reference.
   Result: Still plain text.
   
   7. Updating _IMPORT_MAP inside common-compat
   Updated import map to point to the actual class location.
   This led to several import and LSP errors in sdk.py because it can’t access multiple airflow.sdk modules.
   Result: Build succeeded, but link was still not clickable.
   
   8. Rebuilding after removing all changes
   Removed all previous changes and rebuilt everything cleanly.
   Duplicate BaseOperator definitions appeared again across layers.
   Result: Issue unchanged.
   
   
   Current State
   
   The documentation for all packages builds successfully.
   Only one unrelated warning remains (field list indentation in triggers).
   BaseSensorOperator is still not clickable anywhere in provider documentation.
   
   
   Request for Guidance
   
   
   I’ve spent quite a bit of time on this and tried many different approaches, but I’m still unable to understand why the reference is not resolving. I may be missing something fundamental about how Airflow handles documentation, lazy imports, or cross-references.
   
   It would be really helpful if maintainers or reviewers could guide me on how this is expected to work or what direction I should take next. Any clarification or help here would allow me to move forward and resolve this issue properly.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "Ajay9704 (via GitHub)" <gi...@apache.org>.
Ajay9704 commented on PR #60203:
URL: https://github.com/apache/airflow/pull/60203#issuecomment-3957654338

   I also wanted to add that I’ve already spent quite a bit of time exploring different approaches on my own, and at this point I feel that continuing to try random configurations may not be productive. Instead of going further in that direction, I think it would be more efficient to get some suggestions or guidance from maintainers or reviewers who are familiar with how this part of the documentation system is intended to work.
   
   Any direction or clarification from your side would help me focus on the correct solution path and avoid wasting time on trial-and-error. I’m happy to adjust the approach based on your input and continue working on this once I understand what direction makes the most sense.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "Ajay9704 (via GitHub)" <gi...@apache.org>.
Ajay9704 commented on PR #60203:
URL: https://github.com/apache/airflow/pull/60203#issuecomment-3722571252

   @phanikumv ,Thanks for your patience. I realized that i had missed placing an import at the top of the file earlier, which caused the static check failure. I’ve fixed the import order now and cleaned up the code accordingly.
   
   Could you please re-run the checks when you get a chance?
   
   Thanks a lot for your review and guidance.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "phanikumv (via GitHub)" <gi...@apache.org>.
phanikumv commented on PR #60203:
URL: https://github.com/apache/airflow/pull/60203#issuecomment-3722588152

   > @phanikumv ,Thanks for your patience. I realized that i had missed placing an import at the top of the file earlier, which caused the static check failure. I’ve fixed the import order now and cleaned up the code accordingly.
   > 
   > Could you please re-run the checks when you get a chance?
   > 
   > Thanks a lot for your review and guidance.
   
   done, triggered the re-run


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "phanikumv (via GitHub)" <gi...@apache.org>.
phanikumv commented on PR #60203:
URL: https://github.com/apache/airflow/pull/60203#issuecomment-3722142449

   > @potiuk Sir , sorry for troubling you , but unfortunately my changes didn't worked for static checks , this is my code -intersphinx_mapping["task-sdk"] = (f"[https://airflow.apache.org/docs/task-sdk/{task_sdk_version}/](https://airflow.apache.org/docs/task-sdk/%7Btask_sdk_version%7D/)", (f"[https://airflow.apache.org/docs/task-sdk/{task_sdk_version}/objects.inv](https://airflow.apache.org/docs/task-sdk/%7Btask_sdk_version%7D/objects.inv)",)) .......................................................................... and pre-commit hooks chnaged this to .............................................................................. intersphinx_mapping["task-sdk"] = ( f"[https://airflow.apache.org/docs/task-sdk/{task_sdk_version}/](https://airflow.apache.org/docs/task-sdk/%7Btask_sdk_version%7D/)", (f"[https://airflow.apache.org/docs/task-sdk/{task_sdk_version}/objects.inv](https://airflow.apache.org/docs/task-sdk/%7Btask_sdk_version%7D/objects.inv)",), ) I was unaware about this be
 fore and i apologise for the mistake and now , i have changed that part so now it should also pass static checks , so please if you don't mind re-approve to run all checks again , and if it passes then i can get the screenshots from the docs generated that you asked for from there , please do me this help sir
   
   I have approved the workflows to run now


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Docs: clarify BaseSensorOperator params [airflow]

Posted by "Ajay9704 (via GitHub)" <gi...@apache.org>.
Ajay9704 commented on PR #60203:
URL: https://github.com/apache/airflow/pull/60203#issuecomment-3798184797

   @potiuk I know the update turned out a bit long, but I’ve thoroughly explored all the issues that led to the checks failing and addressed each one.
   
   Please let me know if there’s anything else needed from my side.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org