You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by Asaf Mesika <as...@gmail.com> on 2023/06/27 06:27:32 UTC
Can we archive https://github.com/apache/pulsar-release?
Seems quite neglected
Re: Can we archive https://github.com/apache/pulsar-release?
Posted by Michael Marshall <mm...@apache.org>.
>> Pulsar SQL
>> objections
>It's not objections the blocker, but volunteers.
IIRC, there were objections to moving pulsar sql to its own repo
because there was an effort to contribute it to trino. Given that the
trino contribution has stalled, I think we should move pulsar sql out
of the apache/pulsar code base. It takes a lot of resources to test.
Thanks,
Michael
On Tue, Jun 27, 2023 at 3:26 PM tison <wa...@gmail.com> wrote:
>
> While, I visit tests/integration recently that it can be clear how to
> factor out the testcontainers-based integration utils. Let me file an issue
> to track the work so that we have more visibility about the progress and
> hopefully a nudge to the target - of course separating those code is a good
> idea.
>
> Back to apache/pulsar-release, it seems the separation of connectors is
> part of the prerequisite.
>
> Best,
> tison.
>
>
> tison <wa...@gmail.com> 于2023年6月28日周三 04:15写道：
>
> > > Pulsar SQL
> > > objections
> >
> > It's not objections the blocker, but volunteers.
> >
> > A comparable example is that Flink firstly tried to move out connectors in
> > 2018 and finally made it happen in 2022 (specifically, Kafka connector
> > _today_[1]). That indicates such initiatives require heavy development.
> >
> > For Pulsar SQL separation, there are two major works I don't find time to
> > finish -
> >
> > 1. Factor out integration utils to shared libs and release them. Flink
> > finally has flink-connector-shared-utils[2]
> > and flink-test-utils-junit, flink-connector-test-utils,
> > flink-architecture-tests-test, and so on.
> > 2. Work out a solution to handle Docker image build and release after
> > Pulsar SQL moved out.
> >
> > Best,
> > tison.
> >
> > [1] https://lists.apache.org/thread/vt4746r8kbnmcq7llcsvyjk4gor7gops
> > [2] https://github.com/apache/flink-connector-shared-utils/branches
> >
> >
> > Lari Hotari <lh...@apache.org> 于2023年6月28日周三 01:12写道：
> >
> >> I hope we could finally resume the work on PIP-62 and continue with
> >> the plan that was set in April 2020.
> >> You can find old discussion threads about PIP-62 with this query:
> >> https://lists.apache.org/list?dev@pulsar.apache.org:lte=10y:PIP-62
> >>
> >> There was a discussion in July 2022 about moving Pulsar SQL to a
> >> separate repository. [1]
> >> Could we finally make this happen? Any objections?
> >>
> >> -Lari
> >>
> >> [1] https://lists.apache.org/thread/mflm0pb5235jjk80vol0vs7v0hvowkq8
> >>
> >> On Tue, Jun 27, 2023 at 1:35 PM Enrico Olivelli <eo...@gmail.com>
> >> wrote:
> >> >
> >> > Yes we can.
> >> >
> >> > It was a tentative goal of creating a new workflow to build releases,
> >> > especially for the pulsar-all docker image, that depends both on
> >> > pulsar main repo code and on Pulsar SQL,
> >> > that we wanted to remove from the base repository.
> >> >
> >> > Enrico
> >> >
> >> > Il giorno mar 27 giu 2023 alle ore 08:27 Asaf Mesika
> >> > <as...@gmail.com> ha scritto:
> >> > >
> >> > > Seems quite neglected
> >>
> >
Re: Can we archive https://github.com/apache/pulsar-release?
Posted by tison <wa...@gmail.com>.
While, I visit tests/integration recently that it can be clear how to
factor out the testcontainers-based integration utils. Let me file an issue
to track the work so that we have more visibility about the progress and
hopefully a nudge to the target - of course separating those code is a good
idea.
Back to apache/pulsar-release, it seems the separation of connectors is
part of the prerequisite.
Best,
tison.
tison <wa...@gmail.com> 于2023年6月28日周三 04:15写道：
> > Pulsar SQL
> > objections
>
> It's not objections the blocker, but volunteers.
>
> A comparable example is that Flink firstly tried to move out connectors in
> 2018 and finally made it happen in 2022 (specifically, Kafka connector
> _today_[1]). That indicates such initiatives require heavy development.
>
> For Pulsar SQL separation, there are two major works I don't find time to
> finish -
>
> 1. Factor out integration utils to shared libs and release them. Flink
> finally has flink-connector-shared-utils[2]
> and flink-test-utils-junit, flink-connector-test-utils,
> flink-architecture-tests-test, and so on.
> 2. Work out a solution to handle Docker image build and release after
> Pulsar SQL moved out.
>
> Best,
> tison.
>
> [1] https://lists.apache.org/thread/vt4746r8kbnmcq7llcsvyjk4gor7gops
> [2] https://github.com/apache/flink-connector-shared-utils/branches
>
>
> Lari Hotari <lh...@apache.org> 于2023年6月28日周三 01:12写道：
>
>> I hope we could finally resume the work on PIP-62 and continue with
>> the plan that was set in April 2020.
>> You can find old discussion threads about PIP-62 with this query:
>> https://lists.apache.org/list?dev@pulsar.apache.org:lte=10y:PIP-62
>>
>> There was a discussion in July 2022 about moving Pulsar SQL to a
>> separate repository. [1]
>> Could we finally make this happen? Any objections?
>>
>> -Lari
>>
>> [1] https://lists.apache.org/thread/mflm0pb5235jjk80vol0vs7v0hvowkq8
>>
>> On Tue, Jun 27, 2023 at 1:35 PM Enrico Olivelli <eo...@gmail.com>
>> wrote:
>> >
>> > Yes we can.
>> >
>> > It was a tentative goal of creating a new workflow to build releases,
>> > especially for the pulsar-all docker image, that depends both on
>> > pulsar main repo code and on Pulsar SQL,
>> > that we wanted to remove from the base repository.
>> >
>> > Enrico
>> >
>> > Il giorno mar 27 giu 2023 alle ore 08:27 Asaf Mesika
>> > <as...@gmail.com> ha scritto:
>> > >
>> > > Seems quite neglected
>>
>
Re: Can we archive https://github.com/apache/pulsar-release?
Posted by tison <wa...@gmail.com>.
> Pulsar SQL
> objections
It's not objections the blocker, but volunteers.
A comparable example is that Flink firstly tried to move out connectors in
2018 and finally made it happen in 2022 (specifically, Kafka connector
_today_[1]). That indicates such initiatives require heavy development.
For Pulsar SQL separation, there are two major works I don't find time to
finish -
1. Factor out integration utils to shared libs and release them. Flink
finally has flink-connector-shared-utils[2]
and flink-test-utils-junit, flink-connector-test-utils,
flink-architecture-tests-test, and so on.
2. Work out a solution to handle Docker image build and release after
Pulsar SQL moved out.
Best,
tison.
[1] https://lists.apache.org/thread/vt4746r8kbnmcq7llcsvyjk4gor7gops
[2] https://github.com/apache/flink-connector-shared-utils/branches
Lari Hotari <lh...@apache.org> 于2023年6月28日周三 01:12写道：
> I hope we could finally resume the work on PIP-62 and continue with
> the plan that was set in April 2020.
> You can find old discussion threads about PIP-62 with this query:
> https://lists.apache.org/list?dev@pulsar.apache.org:lte=10y:PIP-62
>
> There was a discussion in July 2022 about moving Pulsar SQL to a
> separate repository. [1]
> Could we finally make this happen? Any objections?
>
> -Lari
>
> [1] https://lists.apache.org/thread/mflm0pb5235jjk80vol0vs7v0hvowkq8
>
> On Tue, Jun 27, 2023 at 1:35 PM Enrico Olivelli <eo...@gmail.com>
> wrote:
> >
> > Yes we can.
> >
> > It was a tentative goal of creating a new workflow to build releases,
> > especially for the pulsar-all docker image, that depends both on
> > pulsar main repo code and on Pulsar SQL,
> > that we wanted to remove from the base repository.
> >
> > Enrico
> >
> > Il giorno mar 27 giu 2023 alle ore 08:27 Asaf Mesika
> > <as...@gmail.com> ha scritto:
> > >
> > > Seems quite neglected
>
Re: Can we archive https://github.com/apache/pulsar-release?
Posted by Lari Hotari <lh...@apache.org>.
I hope we could finally resume the work on PIP-62 and continue with
the plan that was set in April 2020.
You can find old discussion threads about PIP-62 with this query:
https://lists.apache.org/list?dev@pulsar.apache.org:lte=10y:PIP-62
There was a discussion in July 2022 about moving Pulsar SQL to a
separate repository. [1]
Could we finally make this happen? Any objections?
-Lari
[1] https://lists.apache.org/thread/mflm0pb5235jjk80vol0vs7v0hvowkq8
On Tue, Jun 27, 2023 at 1:35 PM Enrico Olivelli <eo...@gmail.com> wrote:
>
> Yes we can.
>
> It was a tentative goal of creating a new workflow to build releases,
> especially for the pulsar-all docker image, that depends both on
> pulsar main repo code and on Pulsar SQL,
> that we wanted to remove from the base repository.
>
> Enrico
>
> Il giorno mar 27 giu 2023 alle ore 08:27 Asaf Mesika
> <as...@gmail.com> ha scritto:
> >
> > Seems quite neglected
Re: Can we archive https://github.com/apache/pulsar-release?
Posted by Enrico Olivelli <eo...@gmail.com>.
Yes we can.
It was a tentative goal of creating a new workflow to build releases,
especially for the pulsar-all docker image, that depends both on
pulsar main repo code and on Pulsar SQL,
that we wanted to remove from the base repository.
Enrico
Il giorno mar 27 giu 2023 alle ore 08:27 Asaf Mesika
<as...@gmail.com> ha scritto:
>
> Seems quite neglected