You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Adaverse (via GitHub)" <gi...@apache.org> on 2023/06/30 23:32:02 UTC
[GitHub] [airflow] Adaverse opened a new issue, #32295: Broken url for the source code point to example_dags under providers
Adaverse opened a new issue, #32295:
URL: https://github.com/apache/airflow/issues/32295
### What do you see as an issue?
When we click on `source code`, we get a 404 page not found
Upon investigation, we see that the docs of all the packages are built in `airflow/providers/<provider_name>/*` except `example_dags`. I suspect it has something to do with the `airflow/example_dags` where we are excluding it in various places during doc builds (my guess).
### Solving the problem
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] amoghrajesh commented on issue #32295: Broken url for the source code pointing to example_dags under providers
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #32295:
URL: https://github.com/apache/airflow/issues/32295#issuecomment-1617702957
(I can take it up too if no one is working on it)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #32295: Broken url for the source code pointing to example_dags under providers
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #32295:
URL: https://github.com/apache/airflow/issues/32295#issuecomment-1621163947
So making any improvements there is most welcome.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] amoghrajesh commented on issue #32295: Broken url for the source code pointing to example_dags under providers
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #32295:
URL: https://github.com/apache/airflow/issues/32295#issuecomment-1655042856
Thanks @Adaverse.
I will migrate it to sys tests. I will need some help in testing it. I will raise a pR today for this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #32295: Broken url for the source code pointing to example_dags under providers
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #32295:
URL: https://github.com/apache/airflow/issues/32295#issuecomment-1621160060
> @potiuk @Adaverse I think this is a bigger problem and cannot be fixed post the docs are generated. This should be fixed in the sphinx config.
>
> Why are we excluding the `example_dags` while building the docs?
>
> ```
> exclude_patterns.extend(
> _get_rst_filepath_from_path(f) for f in pathlib.Path(PACKAGE_DIR).glob("**/example_dags")
> )
> ```
To be honest. I am not sure. This part has always been a bit "trial-and-error" - likely because enabling it causes a number of "index.rst is not part of any TOC". You can remove it and see what efffect it will have. Likely it will need an index to be generated to those.
I do not think we have good set of "reasoning" in the configuration of those docs - in some cases the changes were "simplest way to make the docs build" but without deeper understanding if some of the links there might be missing.
I **think** generally speaking `example_dags` are not considered as public API of the providers, they are just used as "embedded examples" in the docs - and you do not need to have the Python AutoAPI generated documentation, but embedding them using "example include" expects them to be there in the docs.
Just note: example_dags for most cases has been converted to system tests. There are likely few `example_dags` that are left and possibly, the right approach will be to move them to system tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Broken url for the source code pointing to example_dags under providers [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #32295:
URL: https://github.com/apache/airflow/issues/32295#issuecomment-2254368588
This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activity on this issue. Kindly asking to recheck the report against latest Airflow version and let us know if the issue is reproducible. The issue will be closed in next 30 days if no further activity occurs from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] amoghrajesh commented on issue #32295: Broken url for the source code pointing to example_dags under providers
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #32295:
URL: https://github.com/apache/airflow/issues/32295#issuecomment-1617702604
@potiuk if this is an historic issue, it can be fixed in the new post docs script of airflow-site. Wdyt?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #32295: Broken url for the source code pointing to example_dags under providers
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #32295:
URL: https://github.com/apache/airflow/issues/32295#issuecomment-1621163362
And yes. Our docs needs someone to take care about making the whole build and generation process better. That's clear.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] amoghrajesh commented on issue #32295: Broken url for the source code pointing to example_dags under providers
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #32295:
URL: https://github.com/apache/airflow/issues/32295#issuecomment-1653671985
@Adaverse the google system tests are being taken up by @VladaZakharova separately. Any other migrations that you need as part of this issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Adaverse commented on issue #32295: Broken url for the source code pointing to example_dags under providers
Posted by "Adaverse (via GitHub)" <gi...@apache.org>.
Adaverse commented on issue #32295:
URL: https://github.com/apache/airflow/issues/32295#issuecomment-1620773219
> Interesting, the missing source link seems to be not consistent. For Google Cloud Operators and example Cloud Build code in https://airflow.apache.org/docs/apache-airflow-providers-google/stable/operators/cloud/cloud_build.html the links are not broken.
All the source code pointing to `tests.system.providers.google.*` (i.e system tests) or for any providers are being built during doc builds, but the source code in `airflow/providers/<provider_name>/example_dags` aren't being built during doc-builds into HTML. Here the problem is only with the name of the folder i.e `example_dags`. If we rename it to any other name, then it would work.
We are excluding folders with the pattern `*/example_dags/*` in the config of the sphinx. I think there might be the source to this problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #32295: Broken url for the source code pointing to example_dags under providers
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #32295:
URL: https://github.com/apache/airflow/issues/32295#issuecomment-1620030406
Sure. It looks like it is. Needs a bit of investigation where it comes from but likely post-processing the HTML pages is indeed something that might be needed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Adaverse commented on issue #32295: Broken url for the source code pointing to example_dags under providers
Posted by "Adaverse (via GitHub)" <gi...@apache.org>.
Adaverse commented on issue #32295:
URL: https://github.com/apache/airflow/issues/32295#issuecomment-1653956280
Yes @amoghrajesh, been thinking to work on Azure system tests improvements. There are two dags in example_dag folder (I'm the culprit here :)). I have started the works on wasb sensor example dag. There is one more called cosmos document sensor. And also other system test like highlighted here --> #31809. Yet to analyze all the other tests. May be we can do the same exercise on Azure system tests to whatever extent possible but first its important to move the example_dag dags to system test. I have started on wasb sensor part. Feel free to take up the other one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Adaverse commented on issue #32295: Broken url for the source code pointing to example_dags under providers
Posted by "Adaverse (via GitHub)" <gi...@apache.org>.
Adaverse commented on issue #32295:
URL: https://github.com/apache/airflow/issues/32295#issuecomment-1655068131
> Thanks @Adaverse. I will migrate it to sys tests. I will need some help in testing it. I will raise a pR today for this
Sure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] amoghrajesh commented on issue #32295: Broken url for the source code pointing to example_dags under providers
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #32295:
URL: https://github.com/apache/airflow/issues/32295#issuecomment-1621114659
@potiuk @Adaverse I think this is a bigger problem and cannot be fixed post the docs are generated. This should be fixed in the sphinx config.
Why are we excluding the `example_dags` while building the docs?
```
exclude_patterns.extend(
_get_rst_filepath_from_path(f) for f in pathlib.Path(PACKAGE_DIR).glob("**/example_dags")
)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jens-scheffler-bosch commented on issue #32295: Broken url for the source code pointing to example_dags under providers
Posted by "jens-scheffler-bosch (via GitHub)" <gi...@apache.org>.
jens-scheffler-bosch commented on issue #32295:
URL: https://github.com/apache/airflow/issues/32295#issuecomment-1620762996
Interesting, the missing source link seems to be not consistent. For Google Cloud Operators and example Cloud Build code in https://airflow.apache.org/docs/apache-airflow-providers-google/stable/operators/cloud/cloud_build.html the links are not broken.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Adaverse commented on issue #32295: Broken url for the source code pointing to example_dags under providers
Posted by "Adaverse (via GitHub)" <gi...@apache.org>.
Adaverse commented on issue #32295:
URL: https://github.com/apache/airflow/issues/32295#issuecomment-1655068124
> Thanks @Adaverse. I will migrate it to sys tests. I will need some help in testing it. I will raise a pR today for this
Sure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Adaverse commented on issue #32295: Broken url for the source code pointing to example_dags under providers
Posted by "Adaverse (via GitHub)" <gi...@apache.org>.
Adaverse commented on issue #32295:
URL: https://github.com/apache/airflow/issues/32295#issuecomment-1621248854
I think it's better to migrate leftover `example_dags` to system tests and parallelly address the build docs issue and the process overall.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #32295: Broken url for the source code pointing to example_dags under providers
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #32295:
URL: https://github.com/apache/airflow/issues/32295#issuecomment-1621286396
likely yes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Broken url for the source code pointing to example_dags under providers [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #32295: Broken url for the source code pointing to example_dags under providers
URL: https://github.com/apache/airflow/issues/32295
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Broken url for the source code pointing to example_dags under providers [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #32295:
URL: https://github.com/apache/airflow/issues/32295#issuecomment-2322805138
This issue has been closed because it has not received response from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org