回复：[VOTE] PIP-209: Separate C++/Python clients to own repositories

[Baodi Shi <ba...@icloud.com.INVALID>]

+1(non-binding)Baodi ShiThanks2022年9月27日上午2:56，Matteo Merli <mm...@apache.org> 写道：https://github.com/apache/pulsar/issues/17724## MotivationPulsar C++ code base is in the same main repository for the Pulsar project.While the decision was the right one at the time, there is aconsiderable overheadin keeping the C++ client in its current position.### Issues with the current approachThe Pulsar repository has grown a lot in size and number of active developers.1. The frequency of changes in various parts of the codebase hasincreased to a point where the amount of resources dedicated to CI isvery significant. Every change in Java code will trigger the CI jobs for the C++client and every change in the C++ client will do the same. During a CI job we are building the C++ client multiple times: 1. For C++ and Python client tests 2. To build Python wheels to be included in the pulsar Dockerimages (for supporting Pulsar functions)2. The release process for Pulsar has become very complex andrequires building a large number of binaries for C++ and Python clients. This hasbecome too much of a burden during the course of a Pulsar release.## GoalDecouple the development of C++ and Python client libraries from thedevelopment of the core components of Pulsar in Java.## Changes### Repositories1. Move the C++ client code to a new repository`github.com/apache/pulsar-client-c++`2. Move the Python client code to a new repository`github.com/apache/pulsar-client-python`The change will be done without losing any history, extracting asub-directory into a new Git repository.```git filter-repo --subdirectory-filter  pulsar-client-cpp```### Release processThe release process will be split in multiple parts:1. the main Pulsar release will only contain the Java parts (serverdistribution and Java client library)2. The C++ client will have its own release schedule and versioning3. The Python client will have its own release schedule and versioning#### VersioningBoth C++ and Python clients will continue with their own individual versioning.In order to not break anything or cause more confusion, we would needto use a new version that is bigger than the current version (2.11.x).The suggestion is to start the new releases for both C++ and Python from 3.0.0.#### Existing branchesExisting branches of Pulsar, where the C++ client will still be in thesame main the repository and will be receiving bug fixes in theircurrent location.The different location of the new C++ code will make thecherry-picking process slightly more painful in the short term, thoughit will even out in long term.### Projects dependencies#### C++/Python --> PulsarBoth C++ and Python unit/integration tests are designed to run againsta standalone instance of Pulsar broker. In the current form, they'reusing the `master` code that is built to run the tests.After the split, the unit tests will use a Docker image of Pulsar. Wecan use a few different images to test for compatibility1. Latest stable (eg: 2.10.1)2. Nightly (Pulsar Docker image published every day from master branch)#### Pulsar --> PythonTo create a Pulsar image, we are now building the Python client wheelfile and then installing it at build time.Instead, we are going to include a wheel file for a version of thePython client that has been already released.#### Python --> C++The Python client library is just a wrapper on top of the C++ client.Today these are built together, with Python wrapper code residing in asub-directory of C++ client code, and compiled using the same CMakebuild script.By separating the Python client into a different repository, we aregoing to depend on an already released version of the C++ client.#### Automated documentation in the websiteOn the Pulsar website we are auto-generating C++ documentation withthe Doxygen tool and the Python one with Pdoc.Instead of just fetching the main repo code, the website build jobshould be also fetching the new repos to run the tooling.--Matteo Merli<mm...@apache.org>

Re: [VOTE] PIP-209: Separate C++/Python clients to own repositories

[tison <wa...@gmail.com>]

+1 (non-binding)

Best,
tison.


Baodi Shi <ba...@icloud.com.invalid> 于2022年9月27日周二 08:22写道：

> +1(non-binding)
>
> Baodi Shi
> Thanks
>
> 2022年9月27日上午2:56，Matteo Merli <mm...@apache.org> 写道：
>
>
> https://github.com/apache/pulsar/issues/17724
>
>
>
> ## Motivation
>
> Pulsar C++ code base is in the same main repository for the Pulsar project.
>
> While the decision was the right one at the time, there is a
> considerable overhead
> in keeping the C++ client in its current position.
>
> ### Issues with the current approach
>
> The Pulsar repository has grown a lot in size and number of active
> developers.
>
> 1. The frequency of changes in various parts of the codebase has
> increased to a point where the amount of resources dedicated to CI is
> very significant.
>
> Every change in Java code will trigger the CI jobs for the C++
> client and every change in the C++ client will do the same.
>
> During a CI job we are building the C++ client multiple times:
> 1. For C++ and Python client tests
> 2. To build Python wheels to be included in the pulsar Docker
> images (for supporting Pulsar functions)
>
> 2. The release process for Pulsar has become very complex and
> requires building a
> large number of binaries for C++ and Python clients. This has
> become too much of a burden during the course of a Pulsar release.
>
>
> ## Goal
>
> Decouple the development of C++ and Python client libraries from the
> development of the core components of Pulsar in Java.
>
>
> ## Changes
>
> ### Repositories
>
> 1. Move the C++ client code to a new repository
> `github.com/apache/pulsar-client-c++`
> <http://github.com/apache/pulsar-client-c++>
> 2. Move the Python client code to a new repository
> `github.com/apache/pulsar-client-python`
> <http://github.com/apache/pulsar-client-python>
>
> The change will be done without losing any history, extracting a
> sub-directory into a new Git repository.
>
> ```
> git filter-repo --subdirectory-filter pulsar-client-cpp
> ```
>
> ### Release process
>
> The release process will be split in multiple parts:
>
> 1. the main Pulsar release will only contain the Java parts (server
> distribution and Java client library)
> 2. The C++ client will have its own release schedule and versioning
> 3. The Python client will have its own release schedule and versioning
>
> #### Versioning
>
> Both C++ and Python clients will continue with their own individual
> versioning.
>
> In order to not break anything or cause more confusion, we would need
> to use a new version that is bigger than the current version (2.11.x).
>
> The suggestion is to start the new releases for both C++ and Python from
> 3.0.0.
>
>
> #### Existing branches
>
> Existing branches of Pulsar, where the C++ client will still be in the
> same main the repository and will be receiving bug fixes in their
> current location.
>
> The different location of the new C++ code will make the
> cherry-picking process slightly more painful in the short term, though
> it will even out in long term.
>
>
> ### Projects dependencies
>
> #### C++/Python --> Pulsar
>
> Both C++ and Python unit/integration tests are designed to run against
> a standalone instance of Pulsar broker. In the current form, they're
> using the `master` code that is built to run the tests.
>
> After the split, the unit tests will use a Docker image of Pulsar. We
> can use a few different images to test for compatibility
> 1. Latest stable (eg: 2.10.1)
> 2. Nightly (Pulsar Docker image published every day from master branch)
>
> #### Pulsar --> Python
>
> To create a Pulsar image, we are now building the Python client wheel
> file and then installing it at build time.
>
> Instead, we are going to include a wheel file for a version of the
> Python client that has been already released.
>
> #### Python --> C++
>
> The Python client library is just a wrapper on top of the C++ client.
> Today these are built together, with Python wrapper code residing in a
> sub-directory of C++ client code, and compiled using the same CMake
> build script.
>
> By separating the Python client into a different repository, we are
> going to depend on an already released version of the C++ client.
>
>
> #### Automated documentation in the website
>
> On the Pulsar website we are auto-generating C++ documentation with
> the Doxygen tool and the Python one with Pdoc.
>
> Instead of just fetching the main repo code, the website build job
> should be also fetching the new repos to run the tooling.
>
>
> --
> Matteo Merli
> <mm...@apache.org>
>
>
>