[GitHub] [airflow] potiuk commented on a change in pull request #13767: Implement target provider versioning tools

[GitBox <gi...@apache.org>]

potiuk commented on a change in pull request #13767:
URL: https://github.com/apache/airflow/pull/13767#discussion_r560993238



##########
File path: dev/provider_packages/PROVIDER_INDEX_TEMPLATE.rst.jinja2
##########
@@ -0,0 +1,84 @@
+{#
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements.  See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership.  The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied.  See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+ NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE
+ OVERWRITTEN WHEN PREPARING PACKAGES.
+
+ IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
+ `PROVIDER_INDEX_TEMPLATE.rst.jinja2` IN the `provider_packages` DIRECTORY
+
+#}
+
+Package {{ PACKAGE_PIP_NAME }}
+------------------------------------------------------
+
+Release: {{ RELEASE }}{{ VERSION_SUFFIX }}
+
+Provider package
+----------------
+
+This is a provider package for ``{{PROVIDER_PACKAGE_ID}}`` provider. All classes for this provider package
+are in ``{{FULL_PACKAGE_NAME}}`` python package.
+
+Installation
+------------
+
+.. note::
+
+    On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver
+    does not yet work with Apache Airflow and might lead to errors in installation - depends on your choice
+    of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4
+    ``pip install --upgrade pip==20.2.4`` or, in case you use Pip 20.3, you need to add option
+    ``--use-deprecated legacy-resolver`` to your pip install command.
+
+
+You can install this package on top of an existing airflow 2.* installation via
+``pip install {{PACKAGE_PIP_NAME}}``
+{%- if PIP_REQUIREMENTS %}
+
+PIP requirements
+----------------
+
+{{  PIP_REQUIREMENTS_TABLE_RST | safe}}
+{%- endif %}
+{%- if CROSS_PROVIDERS_DEPENDENCIES %}
+
+Cross provider package dependencies
+-----------------------------------
+
+Those are dependencies that might be needed in order to use all the features of the package.
+You need to install the specified backport providers package in order to use them.
+
+You can install such cross-provider dependencies when installing from PyPI. For example:
+
+.. code-block:: bash
+
+    pip install {{ PACKAGE_PIP_NAME }}[{{ CROSS_PROVIDERS_DEPENDENCIES[0] }}]
+
+
+{{ CROSS_PROVIDERS_DEPENDENCIES_TABLE_RST | safe }}
+
+{%- endif %}

Review comment:
       > In my opinion, this page has too much content and some are missing (but the missing ones do not need to be added now). I like the README files. which is available in the documentation of Google libraries. https://googleapis.dev/python/firestore/latest/index.html
   > It is standardized too, but contains all the information you need.
   > 
   > 1. Starting at the top. So I think it's worth adding a description of the package itself.  This information is available in the `provider.yaml` file, although so far the descriptions are not very. precise.h ttps://github.com/apache/airflow/blob/1602ec97c8d5bc7a7a8b42e850ac6c7a7030e47d/airflow/provider.yaml.schema.json#L13-L16
   
   Sure. Happy to add it. This will be easy to incorporate.
   
   > 2. The next step is to link to the documentation, if we want to use the same file in setup.py and in the documentation. This will make it easier to find the documentation when you are in pypi.
   
   I thiink the PyPI and documentation should be different than documentation. PyPI should be more focused on how to install packages and what are the limitations/requirements, but the "full" documentation" should be reached (including the ChangeLog which I think should not be part of README as it will grow over tim). This is how I approached it and I think this is a better idea than to keep the same documentation index and README.
   
   > We have these links in the side menu, but this is quite a new feature and not everyone uses this menu. Especially on mobile devices, I start reading from the package description, not the side menu.
   
   Yeah. I will modify setup.cfg to include few more links (Documentation especially).
    
   > 1. The division into a detailed changelog and a normal changelog is strange for me. As for me, the contents of the ADDITIONAL_INFO.rst file should be a normal documentation page and have the title "Migration guide" because this is exactly what this file contains. As this file is fully human written, I don't think other scripts need to process it as well.
   
   ADDITIONAL_INFO is for backport providers. IMHO this README is something that will evolve evertime. ADDITIONAL_INFO was more about migrating from 1.0 to backports. Effectively CHANGELOG should be written by human as a summary of changes in each version. The assumption is that there will be very rare "migration needed" for those.  Google provider is a bit different than the others, because we migrate python APIs. But in most other packages this will not be a migration guide but human description on what changed in each version. Which I think CHANGELOG is a great name for.
   
   > 2. I think the detailed changelog and migration guides worth moving to new subpages, because it is not important for the new user.  This then will look similar to the Google documentation
   
   Sure. The Detailed guide can be added as a separate page in the docs and linked to. I thought about it as well. Happy to hear why others think but I think we should have this:
   
   * README.md in PyPI describing gerneral "what it is about" + all things important to installing PyPI + link to the Provider "index" documentation
   
   * Provider's index documentation with the same information added + link to detailed commit changelog. It might contain more information (but each Provider is free to add it in the index.rst above the automatically generated part). I do not want to discuss "standard" for this documentation here - we can discuss it separately. In this  change I only want to focus on automatically generated part.
   
   * Detailed changelog - this might be separate page (happy to name it differently if someone has a good proposal).
   
   >    When we move the guides to new pages, we will also be able to easily prepare the migration guide from Airflow 1.10, because we already have a table that facilitates these migrations.
    
   We already have it in Backport providers. I think Provider's documentation should be more "forward" looking rather than focuing in the migration from 1.10. 
   We might instead point out to backport's PYPI readme and mention that for migration from 1.10 people 
   
   > 1. The links in the table use markdown syntax instead of rst.
   
   I think I fixed that, but I can take a look. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org