You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by Sijie Guo <gu...@gmail.com> on 2021/03/12 08:33:13 UTC
[Discuss] Don't include presto/trino in the normal Pulsar distribution
Hi all,
I had a conversation with Jerry and Matteo about NOT including presto/trino
in the normal Pulsar distribution.
Here is the summary of the discussion:
- Move the presto connector to pulsar-presto repo as part of PIP-62
- In the main pulsar repo, we don't bundle presto in the pulsar distribution
- In pulsar-presto repo, we ship it as a standalone connector that can be
installed in an existing trinodb cluster. We can also ship a pulsar-presto
image or distribution which includes trinodb. So people can still easily
test it.
- Update the documentation: People can use pulsar-presto image or
distribution to test presto connector easily; for production deployment, we
focus on documenting how to deploy the connector to an existing trinode
cluster.
Let me know what do you think. If there is no objection, I will move
forward with this plan.
Thanks,
Sijie
Re: [Discuss] Don't include presto/trino in the normal Pulsar distribution
Posted by Sijie Guo <gu...@gmail.com>.
Lari -
We don't change any release cycles. It will be the same release cycle. That
was discussed as part of PIP-62. I can update it to the PIP.
- Sijie
On Mon, Mar 15, 2021 at 2:03 AM Lari Hotari <La...@hotari.net> wrote:
> +1 for this plan. It's good to finally make progress on PIP-62 (
>
> https://github.com/apache/pulsar/wiki/PIP-62%3A-Move-connectors%2C-adapters-and-Pulsar-Presto-to-separate-repositories
> )
> .
>
> I see that one gap in the current PIP-62 document is the lack of relevant
> details. I feel that it is necessary to define how the repositories and
> libraries depend on each other and what are the release cycles and
> versioning strategies (backwards compatibility etc.).
>
> Some questions that could help formulate to the decisions:
> 1)
> a.) What is the release cycle of apache/pulsar-presto repository?
> b.) Is it independent of the release cycle of apache/pulsar?
> c.) What is the versioning scheme of the pulsar-presto repository /
> libraries?
> 2)
> a.) What libraries ("interfaces") does apache/pulsar-presto depend on
> from apache/pulsar?
> b.) What is the release cycle of the apache/pulsar "interfaces" that
> pulsar-presto depends on?
> c.) How are breaking interface changes handled?
> d.) How is backwards compatibility verified?
>
> Could we spend some time to document the decisions on these aspects?
> The benefit of this exercise is that it could help bring up the possible
> issues in dependency, release and version management before implementing
> anything. It would also clarify the details of PIP-62 and make it easier to
> understand the implications in doing Pulsar releases after this change has
> been made.
>
> Sijie, WDYT?
>
> BR, Lari
>
> On Fri, Mar 12, 2021 at 10:33 AM Sijie Guo <gu...@gmail.com> wrote:
>
> > Hi all,
> >
> > I had a conversation with Jerry and Matteo about NOT including
> presto/trino
> > in the normal Pulsar distribution.
> >
> > Here is the summary of the discussion:
> >
> > - Move the presto connector to pulsar-presto repo as part of PIP-62
> > - In the main pulsar repo, we don't bundle presto in the pulsar
> > distribution
> > - In pulsar-presto repo, we ship it as a standalone connector that can be
> > installed in an existing trinodb cluster. We can also ship a
> pulsar-presto
> > image or distribution which includes trinodb. So people can still easily
> > test it.
> > - Update the documentation: People can use pulsar-presto image or
> > distribution to test presto connector easily; for production deployment,
> we
> > focus on documenting how to deploy the connector to an existing trinode
> > cluster.
> >
> > Let me know what do you think. If there is no objection, I will move
> > forward with this plan.
> >
> > Thanks,
> > Sijie
> >
>
Re: [Discuss] Don't include presto/trino in the normal Pulsar distribution
Posted by Lari Hotari <La...@hotari.net>.
+1 for this plan. It's good to finally make progress on PIP-62 (
https://github.com/apache/pulsar/wiki/PIP-62%3A-Move-connectors%2C-adapters-and-Pulsar-Presto-to-separate-repositories)
.
I see that one gap in the current PIP-62 document is the lack of relevant
details. I feel that it is necessary to define how the repositories and
libraries depend on each other and what are the release cycles and
versioning strategies (backwards compatibility etc.).
Some questions that could help formulate to the decisions:
1)
a.) What is the release cycle of apache/pulsar-presto repository?
b.) Is it independent of the release cycle of apache/pulsar?
c.) What is the versioning scheme of the pulsar-presto repository /
libraries?
2)
a.) What libraries ("interfaces") does apache/pulsar-presto depend on
from apache/pulsar?
b.) What is the release cycle of the apache/pulsar "interfaces" that
pulsar-presto depends on?
c.) How are breaking interface changes handled?
d.) How is backwards compatibility verified?
Could we spend some time to document the decisions on these aspects?
The benefit of this exercise is that it could help bring up the possible
issues in dependency, release and version management before implementing
anything. It would also clarify the details of PIP-62 and make it easier to
understand the implications in doing Pulsar releases after this change has
been made.
Sijie, WDYT?
BR, Lari
On Fri, Mar 12, 2021 at 10:33 AM Sijie Guo <gu...@gmail.com> wrote:
> Hi all,
>
> I had a conversation with Jerry and Matteo about NOT including presto/trino
> in the normal Pulsar distribution.
>
> Here is the summary of the discussion:
>
> - Move the presto connector to pulsar-presto repo as part of PIP-62
> - In the main pulsar repo, we don't bundle presto in the pulsar
> distribution
> - In pulsar-presto repo, we ship it as a standalone connector that can be
> installed in an existing trinodb cluster. We can also ship a pulsar-presto
> image or distribution which includes trinodb. So people can still easily
> test it.
> - Update the documentation: People can use pulsar-presto image or
> distribution to test presto connector easily; for production deployment, we
> focus on documenting how to deploy the connector to an existing trinode
> cluster.
>
> Let me know what do you think. If there is no objection, I will move
> forward with this plan.
>
> Thanks,
> Sijie
>
Re: [Discuss] Don't include presto/trino in the normal Pulsar
distribution
Posted by Huanli Meng <hu...@streamnative.io.INVALID>.
+1
BR//Huanli
> On Mar 15, 2021, at 10:08 AM, Guangning E <eg...@gmail.com> wrote:
>
> +1
>
> Jia Zhai <zh...@apache.org> 于2021年3月15日周一 上午10:03写道：
>
>> +1
>> It is clear. and this would make the management of Pulsar repo more easy.
>>
>> Regards.
>>
>> On Fri, Mar 12, 2021 at 11:29 PM Joshua Odmark <jo...@pandio.com> wrote:
>>
>>> +1 I think this is a great idea.
>>>
>>> I like that it simplifies things by making it easier to understand.
>>>
>>> I think it will increase credibility as well. Surprisingly, many people
>>> don’t realize presto/trino is behind the scenes. Making that more well
>> know
>>> will only help. It’s an additional community we can engage as well to
>> help
>>> increase awareness for Pulsar.
>>>
>>> On Fri, Mar 12, 2021 at 12:33 AM Sijie Guo <gu...@gmail.com> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I had a conversation with Jerry and Matteo about NOT including
>>> presto/trino
>>>> in the normal Pulsar distribution.
>>>>
>>>> Here is the summary of the discussion:
>>>>
>>>> - Move the presto connector to pulsar-presto repo as part of PIP-62
>>>> - In the main pulsar repo, we don't bundle presto in the pulsar
>>>> distribution
>>>> - In pulsar-presto repo, we ship it as a standalone connector that can
>> be
>>>> installed in an existing trinodb cluster. We can also ship a
>>> pulsar-presto
>>>> image or distribution which includes trinodb. So people can still
>> easily
>>>> test it.
>>>> - Update the documentation: People can use pulsar-presto image or
>>>> distribution to test presto connector easily; for production
>> deployment,
>>> we
>>>> focus on documenting how to deploy the connector to an existing trinode
>>>> cluster.
>>>>
>>>> Let me know what do you think. If there is no objection, I will move
>>>> forward with this plan.
>>>>
>>>> Thanks,
>>>> Sijie
>>>>
>>>
>>
Re: [Discuss] Don't include presto/trino in the normal Pulsar distribution
Posted by Guangning E <eg...@gmail.com>.
+1
Jia Zhai <zh...@apache.org> 于2021年3月15日周一 上午10:03写道：
> +1
> It is clear. and this would make the management of Pulsar repo more easy.
>
> Regards.
>
> On Fri, Mar 12, 2021 at 11:29 PM Joshua Odmark <jo...@pandio.com> wrote:
>
> > +1 I think this is a great idea.
> >
> > I like that it simplifies things by making it easier to understand.
> >
> > I think it will increase credibility as well. Surprisingly, many people
> > don’t realize presto/trino is behind the scenes. Making that more well
> know
> > will only help. It’s an additional community we can engage as well to
> help
> > increase awareness for Pulsar.
> >
> > On Fri, Mar 12, 2021 at 12:33 AM Sijie Guo <gu...@gmail.com> wrote:
> >
> > > Hi all,
> > >
> > > I had a conversation with Jerry and Matteo about NOT including
> > presto/trino
> > > in the normal Pulsar distribution.
> > >
> > > Here is the summary of the discussion:
> > >
> > > - Move the presto connector to pulsar-presto repo as part of PIP-62
> > > - In the main pulsar repo, we don't bundle presto in the pulsar
> > > distribution
> > > - In pulsar-presto repo, we ship it as a standalone connector that can
> be
> > > installed in an existing trinodb cluster. We can also ship a
> > pulsar-presto
> > > image or distribution which includes trinodb. So people can still
> easily
> > > test it.
> > > - Update the documentation: People can use pulsar-presto image or
> > > distribution to test presto connector easily; for production
> deployment,
> > we
> > > focus on documenting how to deploy the connector to an existing trinode
> > > cluster.
> > >
> > > Let me know what do you think. If there is no objection, I will move
> > > forward with this plan.
> > >
> > > Thanks,
> > > Sijie
> > >
> >
>
Re: [Discuss] Don't include presto/trino in the normal Pulsar distribution
Posted by Jia Zhai <zh...@apache.org>.
+1
It is clear. and this would make the management of Pulsar repo more easy.
Regards.
On Fri, Mar 12, 2021 at 11:29 PM Joshua Odmark <jo...@pandio.com> wrote:
> +1 I think this is a great idea.
>
> I like that it simplifies things by making it easier to understand.
>
> I think it will increase credibility as well. Surprisingly, many people
> don’t realize presto/trino is behind the scenes. Making that more well know
> will only help. It’s an additional community we can engage as well to help
> increase awareness for Pulsar.
>
> On Fri, Mar 12, 2021 at 12:33 AM Sijie Guo <gu...@gmail.com> wrote:
>
> > Hi all,
> >
> > I had a conversation with Jerry and Matteo about NOT including
> presto/trino
> > in the normal Pulsar distribution.
> >
> > Here is the summary of the discussion:
> >
> > - Move the presto connector to pulsar-presto repo as part of PIP-62
> > - In the main pulsar repo, we don't bundle presto in the pulsar
> > distribution
> > - In pulsar-presto repo, we ship it as a standalone connector that can be
> > installed in an existing trinodb cluster. We can also ship a
> pulsar-presto
> > image or distribution which includes trinodb. So people can still easily
> > test it.
> > - Update the documentation: People can use pulsar-presto image or
> > distribution to test presto connector easily; for production deployment,
> we
> > focus on documenting how to deploy the connector to an existing trinode
> > cluster.
> >
> > Let me know what do you think. If there is no objection, I will move
> > forward with this plan.
> >
> > Thanks,
> > Sijie
> >
>
Re: [Discuss] Don't include presto/trino in the normal Pulsar distribution
Posted by Joshua Odmark <jo...@pandio.com>.
+1 I think this is a great idea.
I like that it simplifies things by making it easier to understand.
I think it will increase credibility as well. Surprisingly, many people
don’t realize presto/trino is behind the scenes. Making that more well know
will only help. It’s an additional community we can engage as well to help
increase awareness for Pulsar.
On Fri, Mar 12, 2021 at 12:33 AM Sijie Guo <gu...@gmail.com> wrote:
> Hi all,
>
> I had a conversation with Jerry and Matteo about NOT including presto/trino
> in the normal Pulsar distribution.
>
> Here is the summary of the discussion:
>
> - Move the presto connector to pulsar-presto repo as part of PIP-62
> - In the main pulsar repo, we don't bundle presto in the pulsar
> distribution
> - In pulsar-presto repo, we ship it as a standalone connector that can be
> installed in an existing trinodb cluster. We can also ship a pulsar-presto
> image or distribution which includes trinodb. So people can still easily
> test it.
> - Update the documentation: People can use pulsar-presto image or
> distribution to test presto connector easily; for production deployment, we
> focus on documenting how to deploy the connector to an existing trinode
> cluster.
>
> Let me know what do you think. If there is no objection, I will move
> forward with this plan.
>
> Thanks,
> Sijie
>