[GitHub] [beam] y1chi edited a comment on pull request #13933: [BEAM-11780] Use vendored cloudbuild python client.

[GitBox <gi...@apache.org>]

y1chi edited a comment on pull request #13933:
URL: https://github.com/apache/beam/pull/13933#issuecomment-776327965


   > 1. What is the process to update or regenerate the GCB code? How does one find out What version of GCB used?
   
   we can use 
   ```
   pip install google-apitools
   
   gen_client --discovery_url=cloudbuild.v1 --overwrite --outdir=third_party/py/apache_beam/runners/dataflow/internal/clients/cloudbuild --root_package=. client
   ```
   to generate the client.
   
   I think the version is only bind to the api url version, in this case we are using cloudbuild v1  https://github.com/apache/beam/blob/875e01e45d5bffbb06bde2b92c60c9dec1c383dd/sdks/python/apache_beam/runners/dataflow/internal/clients/cloudbuild/cloudbuild_v1_client.py#L40. If client is updated it will become v1b1, v1b2 etc. 
   > 2. How do we account for GCB dependencies since top-level library is no longer installed ? I am seeing following deps:
   > 
   > ```
   > google-cloud-build==2.0.0
   >   - google-api-core [required: >=1.22.0,<2.0.0dev, installed: 1.26.0]
   >   - libcst [required: >=0.2.5, installed: 0.3.17]
   >   - proto-plus [required: >=0.4.0, installed: 1.13.0]
   > ```
   
   I think if they are solely dependencies for google-cloud-build they won't be needed anymore? Do we need some special handling?
   
   > 
   > It seems strange that we are modifying the external library for internal testing purposes. Can these or similar changes be done upon import instead?
   
   It will add a lot of hacks into the importing process and I'm guessing we were avoiding that before since there are also other gcp client vendored:
   - apache_beam/runners/dataflow/internal/clients/dataflow
   - apache_beam/io/gcp/internal/clients/bigquery
   - apache_beam/io/gcp/internal/clients/storage
   
   I think it's desirable to use the third_party libraries, but it seems to cause a lot of frictions since the dependencies are often out of sync and have different hacks with copybara internally.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org