You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/07/04 05:02:12 UTC
[GitHub] [pulsar-site] michaeljmarshall opened a new pull request, #133: Only generate Javadocs for released versions of Pulsar
michaeljmarshall opened a new pull request, #133:
URL: https://github.com/apache/pulsar-site/pull/133
Main issues: https://github.com/apache/pulsar/issues/14838 https://github.com/apache/pulsar/issues/13916 https://github.com/apache/pulsar/issues/16094
## Motivation
The current Javadocs are generated based on unreleased versions of Pulsar. As a result, the website links have `-SNAPSHOT` in them. This is confusing, and can also lead to problems where the docs do not align with specific versions. This PR is a step towards cleaning up the `-SNAPSHOT` references (there are several more locations, but I am just fixing the Javadoc case in this PR).
## Changes
* Update Javadoc generation script so that it downloads the released source code for pulsar and then builds the Javadocs from that source. (This change is in the first commit.)
* Run the script against versions 2.7.0 through 2.10.1. (These were all added in the second commit.)
## Note
This change does not make these docs referenced on the website. There is additional work that I plan to complete to make the `replace.js` script reference these docs. I will contribute once I fix all of the generated docs with `-SNAPSHOT`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-site] urfreespace commented on pull request #133: Only generate Javadocs for released versions of Pulsar
Posted by GitBox <gi...@apache.org>.
urfreespace commented on PR #133:
URL: https://github.com/apache/pulsar-site/pull/133#issuecomment-1173475539
>
I totally agree, thanks @michaeljmarshall, @Anonymitaet how do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-site] michaeljmarshall commented on pull request #133: Only generate Javadocs for released versions of Pulsar
Posted by GitBox <gi...@apache.org>.
michaeljmarshall commented on PR #133:
URL: https://github.com/apache/pulsar-site/pull/133#issuecomment-1175722892
> I see you've generated API docs for major and minor versions, so for future releases, we need to generate API docs for each version (align with the Pulsar release), correct?
Yes, that is correct. We currently expect the release manager to generate the swagger docs and the python client docs. I think we could make it so the release manager only has to run a single command to generate all docs for the release.
Alternatively, we could probably make some github action that runs when tags are released, but it's probably not worth the effort, and it might lead to minor errors.
> ## Release owner
>
> Currently, API docs are automatically generated.
>
> In this PR, you've changed the automatic workflow to manual work, so who will be responsible for generating API docs? Release managers? If so, would you like to add this required step to the [Pulsar release process - wiki](https://github.com/apache/pulsar/wiki/Release-process)?
The current automated process only updates the latest docs based on the `apache/pulsar` repo's `master` branch. The docs have `-SNAPSHOT` on the end because they are not generated based on the correct tag of the Pulsar code base. While it'd be nice to have a completely automated solution, I think it'll be easier to add a manual solution that relies on running a single script. Once we have the script completely implemented, it might be easier to automate the process.
Additionally, I will update the release process wiki once we have a new process.
For now, I think my primary question is if we can get rid of the broker's Javadocs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-site] michaeljmarshall commented on pull request #133: Only generate Javadocs for released versions of Pulsar
Posted by GitBox <gi...@apache.org>.
michaeljmarshall commented on PR #133:
URL: https://github.com/apache/pulsar-site/pull/133#issuecomment-1173353058
Looks like the generated docs from the broker for all versions are about 304 MB.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-site] michaeljmarshall commented on pull request #133: Only generate Javadocs for released versions of Pulsar
Posted by GitBox <gi...@apache.org>.
michaeljmarshall commented on PR #133:
URL: https://github.com/apache/pulsar-site/pull/133#issuecomment-1182732631
@momo-jun - this PR only fixed the `-SNAPSHOT` issue for Javadocs. The `tools` and the `cpp` client still have `-SNAPSHOT` docs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-site] michaeljmarshall commented on pull request #133: Only generate Javadocs for released versions of Pulsar
Posted by GitBox <gi...@apache.org>.
michaeljmarshall commented on PR #133:
URL: https://github.com/apache/pulsar-site/pull/133#issuecomment-1173351514
@urfreespace - I did not realize just how big these Javadoc files were until opening this PR. This PR literally adds 8 million lines of generated Javadocs. Should we add the script to the main branch and only commit the generated docs to the `asf-site-next` branch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-site] michaeljmarshall commented on pull request #133: Only generate Javadocs for released versions of Pulsar
Posted by GitBox <gi...@apache.org>.
michaeljmarshall commented on PR #133:
URL: https://github.com/apache/pulsar-site/pull/133#issuecomment-1175786907
Note: no docs reference `pulsar.apache.org/api/admin/org/apache/pulsar/client/admin/internal/`, but plenty reference `pulsar.apache.org/api/admin/org/apache/pulsar/client/admin/`, so there shouldn't be any issues replacing the existing pulsar implementation docs. These are all in the `io-use.md` docs files. Note that in the 2.10 docs, these links are fixed to look like `/api/admin/org/apache/pulsar/client/admin/Sink.html#createSink-SinkConfig-java.lang.String-` (they rely on the `replace.js` logic in the `pulsar-site` build scripts). However, the links still don't work for versioned links because we don't generate the right docs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-site] Anonymitaet commented on pull request #133: Only generate Javadocs for released versions of Pulsar
Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on PR #133:
URL: https://github.com/apache/pulsar-site/pull/133#issuecomment-1174534495
@michaeljmarshall thanks for your great contribution! I have several questions:
## Release frequency
I see you've generated API docs for major and minor versions, so for future releases, we need to generate API docs for each version (align with the Pulsar release), correct?
<img width="366" alt="image" src="https://user-images.githubusercontent.com/50226895/177235468-f81b4b7d-1b3c-4221-a334-8d9f140276e6.png">
## Release owner
Currently, API docs are automatically generated every 6 hours.
In this PR, you've changed the automatic workflow to manual work, so who will be responsible for generating API docs?
Release managers? If so, would you like to add this required step to the [Pulsar release process - wiki](https://github.com/apache/pulsar/wiki/Release-process)?
~~~
FYI @D-2-Ed @DaveDuggins @momo-jun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-site] michaeljmarshall commented on pull request #133: Only generate Javadocs for released versions of Pulsar
Posted by GitBox <gi...@apache.org>.
michaeljmarshall commented on PR #133:
URL: https://github.com/apache/pulsar-site/pull/133#issuecomment-1173363143
Interestingly, the following returns now results:
```shell
grep -R "/api/pulsar-broker" content/docs/
```
That indicates to me that the website doesn't actually point to the broker's Javadocs. Why do we generate and host them if they are not referenced? I wonder if we can remove them and save some space.
Either way, we still need to decide the right workflow for adding generated docs. I think it would make sense to keep them in the `main` branch, but I am open to opinions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-site] michaeljmarshall commented on pull request #133: Only generate Javadocs for released versions of Pulsar
Posted by GitBox <gi...@apache.org>.
michaeljmarshall commented on PR #133:
URL: https://github.com/apache/pulsar-site/pull/133#issuecomment-1175784198
I just discovered that the Pulsar Admin API Javadocs changed their generation mechanism starting with 2.8.0 because of this change: https://github.com/apache/pulsar/pull/9246. In my two most recent commits, I fix the Javadoc generation script. This fix takes care of a primary issue where the docs reference links like `https://pulsar.apache.org/api/admin/2.9.0-SNAPSHOT/org/apache/pulsar/client/admin/Sink.html#deleteSink-java.lang.String-java.lang.String-java.lang.String-`, but the links do not work unless they are `https://pulsar.apache.org/api/admin/org/apache/pulsar/client/admin/Sink.html#deleteSink-java.lang.String-java.lang.String-java.lang.String-`.
By generating the docs based on the interfaces, not the `Impl` classes, we'll get the right docs and the links should be restored.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-site] urfreespace commented on pull request #133: Only generate Javadocs for released versions of Pulsar
Posted by GitBox <gi...@apache.org>.
urfreespace commented on PR #133:
URL: https://github.com/apache/pulsar-site/pull/133#issuecomment-1173476143
> Interestingly, the following returns no results when run on the `asf-site-next` branch for this repo:
>
> ```shell
> grep -R "/api/pulsar-broker" content/docs/
> ```
>
> That indicates to me that the website doesn't actually point to the broker's Javadocs. Why do we generate and host them if they are not referenced? I wonder if we can remove them and save some space.
>
> Either way, we still need to decide the right workflow for adding generated docs. I think it would make sense to keep them in the `main` branch, but I am open to opinions.
I totally agree, thanks @michaeljmarshall, @Anonymitaet how do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-site] Anonymitaet commented on pull request #133: Only generate Javadocs for released versions of Pulsar
Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on PR #133:
URL: https://github.com/apache/pulsar-site/pull/133#issuecomment-1174535186
FYI
@D-2-Ed @DaveDuggins
@SignorMercurio @momo-jun: this PR removes `SNAPSHOT`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-site] michaeljmarshall commented on pull request #133: Only generate Javadocs for released versions of Pulsar
Posted by GitBox <gi...@apache.org>.
michaeljmarshall commented on PR #133:
URL: https://github.com/apache/pulsar-site/pull/133#issuecomment-1175745654
I updated the PR to remove the `pulsar-broker` doc generation. I started a discussion on the dev@ mailing list to propose removing these docs from the website permanently.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-site] michaeljmarshall merged pull request #133: Only generate Javadocs for released versions of Pulsar
Posted by GitBox <gi...@apache.org>.
michaeljmarshall merged PR #133:
URL: https://github.com/apache/pulsar-site/pull/133
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-site] momo-jun commented on pull request #133: Only generate Javadocs for released versions of Pulsar
Posted by GitBox <gi...@apache.org>.
momo-jun commented on PR #133:
URL: https://github.com/apache/pulsar-site/pull/133#issuecomment-1182700149
Hi @michaeljmarshall did your PR fix the reported three issues mentioned above? They are still in an open state. Not sure whether you will have follow-up fixes or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org