You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by 吴治国 <wz...@163.com> on 2022/08/16 09:48:34 UTC
RE: Re: [Discuss] Initial Draft for Roadmap
Hi community,
In my original idea, I was planning to provide Bigtop3.1.0 + Ambari2.7.5 for users at first, and keep all big changes in Ambari2.8.0, e.g.
1、Fix some security problems by upgrading Spring Framework and others
2、Frontend refactor
3、Upgrade python version
etc.
All these changes will take long time, which may be bad for the release cycle of Ambari itself.
So I'm considering we can move Mpack development to Ambari side, and 2.8.0 is all about Bigtop stack, move other big changes to next major version(except the security problem, which absolutely need to be fixed in next release).
But we need to make final decision for the things we discussed before, like,
1、Add prefix to installation path(/usr/bigtop)
2、Which repo to store the packages(with prefix in installation path)?
3、Related name(Stack name: BIGTOP, scripts: conf-select, distro-select, bigtop-select are all ok for me)?
Any other things I forgot?
What’s your opinion?
Best Regards,
Zhiguo Wu
On 2022/06/23 06:26:44 "Battula, Brahma Reddy" wrote:
>
> IMO, First way is not heavier when we consider all the efforts for mpack with upgrades. And first one is easiest to adopt now.
>
> We'll start discussing the bigtop mailing list further.
>
>
> On 21/06/22, 1:11 PM, "吴治国" <wz...@163.com> wrote:
>
> Thanks for the details, Brahma and Kengo.
>
> If I understand it correctly, there are two approaches under discussion:
>
> 1、Prepare to release 2.7.7, in this version, we won't use HDP components anymore, we'll replace it with the community version provided by Bigtop, and update RPM and DEB build scripts provided by Bigtop, change the installation path so hdp-select can be reused. (Due to component version changes, I don't know if the code under /stacks needs to be updated)
> 2、Develop Bigtop mpack for Ambari 2.7.5, we don't need to change Ambari's code (if the code under /stacks can be reused in the first way, maybe it also works in this way?)
>
> Isn't the workload of the first way heavier?
>
>
> About the meeting
>
> I thought maybe email discussion would be better?
> The discussion can be open for a few days and all interested people in the community can participate, not just a few of us.
> And due to time zone and language issues, many people are hard to attending the meetings, email discussions may be more suitable for these people.
>
> What do you think?
>
> Best Regards,
> Zhiguo Wu
>
> > On Jun 20, 2022, at 23:15, Battula, Brahma Reddy <bb...@visa.com.INVALID> wrote:
> >
> >>> Assuming the understanding above is correct, my new concerns are:
> >
> > Yes, Hope we'll in same page now. Still we can discuss in weekly call.
> >
> >>> * Can we still call it as "HDP"? Doesn't it infringe Cloudera's trademark?
> >>> If so, should we give a new stack name to it, as you mentioned as
> >>> planA in [3]?
> >>> (or it makes upgrading HDP cluster difficult? I don't understand it so much)
> >
> > Yes, it may be. Hence I introduced the planA and preferred it. one more idea(bad) came in my mind that can we treat "HDP: Hadoop Data Platform", not sure whether it should be ok or not(and we might not delivering as distro's).
> >
> >>> * It may be a bit tough work to fix Bigtop's build scripts so that its
> >>> artifacts match Ambari's requirements.
> >>> (package versioning, file layouts, included and not included files,
> >>> file permissions, user/group creation, etc.)
> >
> > Only paths matter right everything else can be re-used from the hdp-select.
> >
> >
> > On 20/06/22, 8:13 PM, "Kengo Seki" <sekikn@apache.org <ma...@apache.org>> wrote:
> >
> > Thanks for the elaboration Brahma, but I'm a bit confused.
> > At this point, my understanding is as follows. Are these correct?
> >
> > * In the 2.7.x line, we'll still use a stack definition based on HDP's one [1].
> >
> > * But regarding RPM and DEB packages, we'll build them from source by leveraging
> > Bigtop's build scripts and publish them on the ASF distribution site
> > or somewhere.
> > repoinfo.xml [2] will also be fixed to point that repository.
> >
> > Assuming the understanding above is correct, my new concerns are:
> >
> > * Can we still call it as "HDP"? Doesn't it infringe Cloudera's trademark?
> > If so, should we give a new stack name to it, as you mentioned as
> > planA in [3]?
> > (or it makes upgrading HDP cluster difficult? I don't understand it so much)
> >
> > * It may be a bit tough work to fix Bigtop's build scripts so that its
> > artifacts match Ambari's requirements.
> > (package versioning, file layouts, included and not included files,
> > file permissions, user/group creation, etc.)
> > I'm not sure if it's faster than developing Mpack.
> >
> > [1]: https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dFX07wfIY8aukfEUcJe9GN0rds8t3%2Fpenmsiszglkn4%3D&reserved=0 <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dFX07wfIY8aukfEUcJe9GN0rds8t3%2Fpenmsiszglkn4%3D&reserved=0>
> > [2]: https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP%2F2.6%2Frepos%2Frepoinfo.xml&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MDz57IloHfWYWcwmAI9h1pYDG45v1UbU6K2M%2BkpcDEA%3D&reserved=0 <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP%2F2.6%2Frepos%2Frepoinfo.xml&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MDz57IloHfWYWcwmAI9h1pYDG45v1UbU6K2M%2BkpcDEA%3D&reserved=0>
> > [3]: https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm77lhzo0njr2dhoock5twtm0c19j2py8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=P1pPHwM1YggdPT9qD4aAvJdehjjZdZ8mLhvn0ZfR7Gw%3D&reserved=0 <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm77lhzo0njr2dhoock5twtm0c19j2py8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=P1pPHwM1YggdPT9qD4aAvJdehjjZdZ8mLhvn0ZfR7Gw%3D&reserved=0>
> >
> > Kengo Seki <sekikn@gmail.com <ma...@gmail.com>>
> >
> > On Mon, Jun 20, 2022 at 8:05 PM Battula, Brahma Reddy
> > <bbattula@visa.com.invalid <ma...@visa.com.invalid>> wrote:
> >>
> >> We'll not use the HDP Packages (Only selects we'll use from hdp, which can be plugged in bigtop). Still we'll use opensource packages and bigtop to create the RPM's and selects.
> >> Just we can use the source tar balls opensource components(e.g Hadoop-3.2.3 source tarballs from apache repo) and build rpm's and publish in following repo. Or from the apache github repo's.
> >>
> >> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2Fbigtop%2Fbigtop-3.1.0%2Frepos%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RyNyy7fOUlWBSFMJdd1iRMrmlKOVyA5NEGuMpSkCSEs%3D&reserved=0 <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2Fbigtop%2Fbigtop-3.1.0%2Frepos%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RyNyy7fOUlWBSFMJdd1iRMrmlKOVyA5NEGuMpSkCSEs%3D&reserved=0>
> >>
> >> Planning to have sync up call on this week, once it's finalize, we can summarise here the approach. What do you think..?
> >>
> >>
> >> On 20/06/22, 4:06 PM, "Kengo Seki" <sekikn@apache.org <ma...@apache.org>> wrote:
> >>
> >> Thanks for the reply, Brahma!
> >>
> >>> We'll use following repo, how other components in Apache are hosted.
> >>
> >> Using that site for distribution has no problem. My concern is, do we
> >> have the source tree for building HDP packages and does it still work
> >> without the artifacts behind the paywall?
> >> According to ASF's release policy [1], we must publish the
> >> corresponding source releases whenever we publish binary artifacts,
> >> such as RPM and DEB packages for HDP.
> >>
> >> [1]: https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Flegal%2Frelease-policy.html%23compiled-packages&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qPEbEEpIN8NCsrMnxW5xKbB%2B19bDJprq%2FiXjIF09vls%3D&reserved=0 <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Flegal%2Frelease-policy.html%23compiled-packages&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qPEbEEpIN8NCsrMnxW5xKbB%2B19bDJprq%2FiXjIF09vls%3D&reserved=0>
> >>
> >> Kengo Seki <sekikn@apache.org <ma...@apache.org>>
> >>
> >> On Mon, Jun 20, 2022 at 6:19 PM Battula, Brahma Reddy
> >> <bbattula@visa.com.invalid <ma...@visa.com.invalid>> wrote:
> >>>
> >>> Hi Kengo Seki,
> >>>
> >>> Thanks for pitching here.. We'll use following repo, how other components in Apache are hosted. Any other suggestions..?
> >>>
> >>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OUGhE45KsDvPum4%2Fd9cwlJPfH66OlKE%2BgD557%2BiiAC8%3D&reserved=0 <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OUGhE45KsDvPum4%2Fd9cwlJPfH66OlKE%2BgD557%2BiiAC8%3D&reserved=0>
> >>>
> >>>
> >>>
> >>> On 20/06/22, 2:32 PM, "Kengo Seki" <sekikn@apache.org <ma...@apache.org>> wrote:
> >>>
> >>> Oh, I've just noticed that I submitted an additional question into the
> >>> wrong thread mistakenly.
> >>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fj6kc2h1lphvjvsblcthcl5yzzmjk3tb5&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UTDhFu1Bloa3FzD0shWWHoDt%2FrQD22JziV1%2BNAWp8NQ%3D&reserved=0 <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fj6kc2h1lphvjvsblcthcl5yzzmjk3tb5&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UTDhFu1Bloa3FzD0shWWHoDt%2FrQD22JziV1%2BNAWp8NQ%3D&reserved=0> is
> >>> originally intended as a reply to this thread. Sorry for the
> >>> confusion.
> >>>
> >>> Kengo Seki <sekikn@apache.org <ma...@apache.org>>
> >>>
> >>> On Mon, Jun 20, 2022 at 11:31 AM Kengo Seki <sekikn@apache.org <ma...@apache.org>> wrote:
> >>>>
> >>>> Thank you so much for writing the draft and commenting on it, Brahma and Viraj!
> >>>> Firstly, a minor comment:
> >>>>
> >>>>> Trunk Code : Parallel we can work for the trunk code, this we might give as 2.8.
> >>>>> * Working on bigtop mpack
> >>>>
> >>>> This means adding a new built-in stack which uses the Bigtop
> >>>> repository, and not developing Mpack on the Bigtop side, right?
> >>>> If so, "bigtop mpack" sounds confusing a bit.
> >>>>
> >>>>> hdp-select can stay on branch-2.7 and it can be moved to bigtop for trunk branch. Thoughts?
> >>>>
> >>>> Sounds good to me. During the 2.7.x releases, users can use the HDP
> >>>> stack by default. In addition, they can also add the Bigtop stack
> >>>> through the Mpack, as Zhiguo mentioned in [1].
> >>>> Since 2.8 or 3.0, we will drop HDP and users can use the built-in
> >>>> Bigtop stack by default. It also allows us to drop the Mpack on the
> >>>> Bigtop side.
> >>>>
> >>>> Regarding the built-in Bigtop stack developed on trunk, the current
> >>>> BIGTOP stack in Ambari [2] may be a good start point to be upgraded.
> >>>> I ensured that Ambari 2.7.6 was successfully built with the
> >>>> `-Dstack.distribution=BIGTOP` option, though Bigtop 0.8 is too old to
> >>>> use for now.
> >>>>
> >>>> [1]: https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iqQ6QCM1%2BMYJ61tU%2FFk%2BWC8GvftYYGhrnu6V%2FFJ3d7o%3D&reserved=0 <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iqQ6QCM1%2BMYJ61tU%2FFk%2BWC8GvftYYGhrnu6V%2FFJ3d7o%3D&reserved=0>
> >>>> [2]: https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=FR5NTHCZt%2FQcqy9NyT63Q3NIYlEeJ6sIpbOqzfDOMRY%3D&reserved=0 <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
[message truncated...]
Re: [Discuss] Initial Draft for Roadmap
Posted by 吴治国 <wz...@163.com>.
Hi 哲远,
Thanks for your suggestion, I'm also considering add those components in the future.
But now we should focus on next release.
BTW, ClickHouse is under discussion, I’ll start working on it soon or later if we all agree to include it.
Best Regards,
Zhiguo Wu
> On Aug 22, 2022, at 11:31, 王哲远 <wa...@gmail.com> wrote:
>
> I think we should add the data lake(Delta Lake, Hudi or Iceberg) to the
> stack for the next release and OLAP(ClickHouse or Doris) for
> future releases.
>
> 吴治国 <wz...@163.com> 于2022年8月17日周三 15:20写道：
>
>> Hi Jun,
>>
>> Sorry I miss some details here.
>>
>> After we move Bigtop Ambari MPack to Ambari side, it's no longer a
>> management-pack[1] for Ambari.
>> It'll be the default stack[2] which will replace HDP while compiling[3],
>> also, we can remove MPack code on Bigtop side[4], you can find old version
>> of bigtop stack in Ambari release 2.7.6[5].
>>
>> The reason I bring up Frontend Refactor is that the framework is outdated,
>> which cause it difficult to add new features, it also has compiled issue in
>> China, but the work won't be done in release 2.8.0, we can discuss in the
>> future.
>>
>> In addition, the development of components provided by Bigtop 3.2.0 is
>> almost done[6], I think we only need three more components(Flink, Spark3
>> and Phoenix, maybe Alluxio also needed?) to prepare the next release, but
>> there are some code still need to be updated after Bigtop provide the
>> packages with prefix in the installation path(which is important for stack
>> upgrade).
>>
>> Best Regards,
>> Zhiguo Wu
>>
>> [1]: https://github.com/apache/ambari/tree/trunk/contrib/management-packs
>> [2]:
>> https://github.com/apache/ambari/tree/trunk/ambari-server/src/main/resources/stacks
>> [3]: https://github.com/apache/ambari/blob/trunk/pom.xml#L90
>> [4]:
>> https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/bigtop-ambari-mpack
>> [5]:
>> https://github.com/apache/ambari/tree/release-2.7.6/ambari-server/src/main/resources/stacks
>> [6]:
>> https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/bigtop-ambari-mpack/bgtp-ambari-mpack/src/main/resources/stacks/BGTP/1.0/services
>>
>>> On Aug 17, 2022, at 11:08, Jun HE <ju...@apache.org> wrote:
>>>
>>> I agree with "1. Fix some security problems by upgrading Spring Framework
>>> and others".Security is important and we can use this as a ramp-up as the
>>> community is gradually bringing everything in Ambari back on track, like
>> PR
>>> review, CI jobs,...
>>>
>>> For other things like "frontend refactor", it would be good to have more
>>> inputs on existing issues/limitations before we decide to "refactor".
>>> For the mpack part, my understanding is that "management packs" is a
>>> framework that allows different stack vendors to provide a compatible
>> stack
>>> that can be maintained and operated by Ambari. There could be minor
>>> differences between different vendors. So the definition like
>>> path/repo/name etc should be the vendor's decision instead of being
>> defined
>>> by Ambari. I would suggest leaving the mpack related work to a certain
>>> community, for example, Bigtop, and Ambari just review and accept their
>> PR
>>> just for the user's convenience.
>>>
>>> Just my 2 cents. And happy to learn any comments.
>>>
>>> 吴治国 <wz...@163.com> 于2022年8月16日周二 17:50写道：
>>>
>>>> Hi community,
>>>>
>>>> In my original idea, I was planning to provide Bigtop3.1.0 + Ambari2.7.5
>>>> for users at first, and keep all big changes in Ambari2.8.0, e.g.
>>>> 1、Fix some security problems by upgrading Spring Framework and others
>>>> 2、Frontend refactor
>>>> 3、Upgrade python version
>>>> etc.
>>>>
>>>> All these changes will take long time, which may be bad for the release
>>>> cycle of Ambari itself.
>>>> So I'm considering we can move Mpack development to Ambari side, and
>> 2.8.0
>>>> is all about Bigtop stack, move other big changes to next major
>>>> version(except the security problem, which absolutely need to be fixed
>> in
>>>> next release).
>>>>
>>>> But we need to make final decision for the things we discussed before,
>>>> like,
>>>> 1、Add prefix to installation path(/usr/bigtop)
>>>> 2、Which repo to store the packages(with prefix in installation path)?
>>>> 3、Related name(Stack name: BIGTOP, scripts: conf-select, distro-select,
>>>> bigtop-select are all ok for me)?
>>>> Any other things I forgot?
>>>>
>>>> What’s your opinion?
>>>>
>>>> Best Regards,
>>>> Zhiguo Wu
>>>>
>>>> On 2022/06/23 06:26:44 "Battula, Brahma Reddy" wrote:
>>>>>
>>>>> IMO, First way is not heavier when we consider all the efforts for
>> mpack
>>>> with upgrades. And first one is easiest to adopt now.
>>>>>
>>>>> We'll start discussing the bigtop mailing list further.
>>>>>
>>>>>
>>>>> On 21/06/22, 1:11 PM, "吴治国" <wz...@163.com> wrote:
>>>>>
>>>>> Thanks for the details, Brahma and Kengo.
>>>>>
>>>>> If I understand it correctly, there are two approaches under
>>>> discussion:
>>>>>
>>>>> 1、Prepare to release 2.7.7, in this version, we won't use HDP
>>>> components anymore, we'll replace it with the community version
>> provided by
>>>> Bigtop, and update RPM and DEB build scripts provided by Bigtop, change
>> the
>>>> installation path so hdp-select can be reused. (Due to component version
>>>> changes, I don't know if the code under /stacks needs to be updated)
>>>>> 2、Develop Bigtop mpack for Ambari 2.7.5, we don't need to change
>>>> Ambari's code (if the code under /stacks can be reused in the first way,
>>>> maybe it also works in this way?)
>>>>>
>>>>> Isn't the workload of the first way heavier?
>>>>>
>>>>>
>>>>> About the meeting
>>>>>
>>>>> I thought maybe email discussion would be better?
>>>>> The discussion can be open for a few days and all interested people
>>>> in the community can participate, not just a few of us.
>>>>> And due to time zone and language issues, many people are hard to
>>>> attending the meetings, email discussions may be more suitable for these
>>>> people.
>>>>>
>>>>> What do you think?
>>>>>
>>>>> Best Regards,
>>>>> Zhiguo Wu
>>>>>
>>>>>> On Jun 20, 2022, at 23:15, Battula, Brahma Reddy
>>>> <bb...@visa.com.INVALID> wrote:
>>>>>>
>>>>>>>> Assuming the understanding above is correct, my new concerns are:
>>>>>>
>>>>>> Yes, Hope we'll in same page now. Still we can discuss in weekly
>>>> call.
>>>>>>
>>>>>>>> * Can we still call it as "HDP"? Doesn't it infringe Cloudera's
>>>> trademark?
>>>>>>>> If so, should we give a new stack name to it, as you mentioned as
>>>>>>>> planA in [3]?
>>>>>>>> (or it makes upgrading HDP cluster difficult? I don't understand
>>>> it so much)
>>>>>>
>>>>>> Yes, it may be. Hence I introduced the planA and preferred it. one
>>>> more idea(bad) came in my mind that can we treat "HDP: Hadoop Data
>>>> Platform", not sure whether it should be ok or not(and we might not
>>>> delivering as distro's).
>>>>>>
>>>>>>>> * It may be a bit tough work to fix Bigtop's build scripts so
>>>> that its
>>>>>>>> artifacts match Ambari's requirements.
>>>>>>>> (package versioning, file layouts, included and not included
>>>> files,
>>>>>>>> file permissions, user/group creation, etc.)
>>>>>>
>>>>>> Only paths matter right everything else can be re-used from the
>>>> hdp-select.
>>>>>>
>>>>>>
>>>>>> On 20/06/22, 8:13 PM, "Kengo Seki" <sekikn@apache.org <
>>>> ma...@apache.org>> wrote:
>>>>>>
>>>>>> Thanks for the elaboration Brahma, but I'm a bit confused.
>>>>>> At this point, my understanding is as follows. Are these correct?
>>>>>>
>>>>>> * In the 2.7.x line, we'll still use a stack definition based on
>>>> HDP's one [1].
>>>>>>
>>>>>> * But regarding RPM and DEB packages, we'll build them from source
>>>> by leveraging
>>>>>> Bigtop's build scripts and publish them on the ASF distribution
>>>> site
>>>>>> or somewhere.
>>>>>> repoinfo.xml [2] will also be fixed to point that repository.
>>>>>>
>>>>>> Assuming the understanding above is correct, my new concerns are:
>>>>>>
>>>>>> * Can we still call it as "HDP"? Doesn't it infringe Cloudera's
>>>> trademark?
>>>>>> If so, should we give a new stack name to it, as you mentioned as
>>>>>> planA in [3]?
>>>>>> (or it makes upgrading HDP cluster difficult? I don't understand
>>>> it so much)
>>>>>>
>>>>>> * It may be a bit tough work to fix Bigtop's build scripts so that
>>>> its
>>>>>> artifacts match Ambari's requirements.
>>>>>> (package versioning, file layouts, included and not included files,
>>>>>> file permissions, user/group creation, etc.)
>>>>>> I'm not sure if it's faster than developing Mpack.
>>>>>>
>>>>>> [1]:
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dFX07wfIY8aukfEUcJe9GN0rds8t3%2Fpenmsiszglkn4%3D&reserved=0
>>>> <
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dFX07wfIY8aukfEUcJe9GN0rds8t3%2Fpenmsiszglkn4%3D&reserved=0
>>>>>
>>>>>> [2]:
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP%2F2.6%2Frepos%2Frepoinfo.xml&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MDz57IloHfWYWcwmAI9h1pYDG45v1UbU6K2M%2BkpcDEA%3D&reserved=0
>>>> <
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP%2F2.6%2Frepos%2Frepoinfo.xml&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MDz57IloHfWYWcwmAI9h1pYDG45v1UbU6K2M%2BkpcDEA%3D&reserved=0
>>>>>
>>>>>> [3]:
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm77lhzo0njr2dhoock5twtm0c19j2py8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=P1pPHwM1YggdPT9qD4aAvJdehjjZdZ8mLhvn0ZfR7Gw%3D&reserved=0
>>>> <
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm77lhzo0njr2dhoock5twtm0c19j2py8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=P1pPHwM1YggdPT9qD4aAvJdehjjZdZ8mLhvn0ZfR7Gw%3D&reserved=0
>>>>>
>>>>>>
>>>>>> Kengo Seki <sekikn@gmail.com <ma...@gmail.com>>
>>>>>>
>>>>>> On Mon, Jun 20, 2022 at 8:05 PM Battula, Brahma Reddy
>>>>>> <bbattula@visa.com.invalid <ma...@visa.com.invalid>> wrote:
>>>>>>>
>>>>>>> We'll not use the HDP Packages (Only selects we'll use from hdp,
>>>> which can be plugged in bigtop). Still we'll use opensource packages and
>>>> bigtop to create the RPM's and selects.
>>>>>>> Just we can use the source tar balls opensource components(e.g
>>>> Hadoop-3.2.3 source tarballs from apache repo) and build rpm's and
>> publish
>>>> in following repo. Or from the apache github repo's.
>>>>>>>
>>>>>>>
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2Fbigtop%2Fbigtop-3.1.0%2Frepos%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RyNyy7fOUlWBSFMJdd1iRMrmlKOVyA5NEGuMpSkCSEs%3D&reserved=0
>>>> <
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2Fbigtop%2Fbigtop-3.1.0%2Frepos%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RyNyy7fOUlWBSFMJdd1iRMrmlKOVyA5NEGuMpSkCSEs%3D&reserved=0
>>>>>
>>>>>>>
>>>>>>> Planning to have sync up call on this week, once it's finalize,
>>>> we can summarise here the approach. What do you think..?
>>>>>>>
>>>>>>>
>>>>>>> On 20/06/22, 4:06 PM, "Kengo Seki" <sekikn@apache.org <
>>>> ma...@apache.org>> wrote:
>>>>>>>
>>>>>>> Thanks for the reply, Brahma!
>>>>>>>
>>>>>>>> We'll use following repo, how other components in Apache are
>>>> hosted.
>>>>>>>
>>>>>>> Using that site for distribution has no problem. My concern is,
>>>> do we
>>>>>>> have the source tree for building HDP packages and does it still
>>>> work
>>>>>>> without the artifacts behind the paywall?
>>>>>>> According to ASF's release policy [1], we must publish the
>>>>>>> corresponding source releases whenever we publish binary
>>>> artifacts,
>>>>>>> such as RPM and DEB packages for HDP.
>>>>>>>
>>>>>>> [1]:
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Flegal%2Frelease-policy.html%23compiled-packages&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qPEbEEpIN8NCsrMnxW5xKbB%2B19bDJprq%2FiXjIF09vls%3D&reserved=0
>>>> <
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Flegal%2Frelease-policy.html%23compiled-packages&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qPEbEEpIN8NCsrMnxW5xKbB%2B19bDJprq%2FiXjIF09vls%3D&reserved=0
>>>>>
>>>>>>>
>>>>>>> Kengo Seki <sekikn@apache.org <ma...@apache.org>>
>>>>>>>
>>>>>>> On Mon, Jun 20, 2022 at 6:19 PM Battula, Brahma Reddy
>>>>>>> <bbattula@visa.com.invalid <ma...@visa.com.invalid>> wrote:
>>>>>>>>
>>>>>>>> Hi Kengo Seki,
>>>>>>>>
>>>>>>>> Thanks for pitching here.. We'll use following repo, how other
>>>> components in Apache are hosted. Any other suggestions..?
>>>>>>>>
>>>>>>>>
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OUGhE45KsDvPum4%2Fd9cwlJPfH66OlKE%2BgD557%2BiiAC8%3D&reserved=0
>>>> <
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OUGhE45KsDvPum4%2Fd9cwlJPfH66OlKE%2BgD557%2BiiAC8%3D&reserved=0
>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 20/06/22, 2:32 PM, "Kengo Seki" <sekikn@apache.org <
>>>> ma...@apache.org>> wrote:
>>>>>>>>
>>>>>>>> Oh, I've just noticed that I submitted an additional question
>>>> into the
>>>>>>>> wrong thread mistakenly.
>>>>>>>>
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fj6kc2h1lphvjvsblcthcl5yzzmjk3tb5&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UTDhFu1Bloa3FzD0shWWHoDt%2FrQD22JziV1%2BNAWp8NQ%3D&reserved=0
>>>> <
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fj6kc2h1lphvjvsblcthcl5yzzmjk3tb5&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UTDhFu1Bloa3FzD0shWWHoDt%2FrQD22JziV1%2BNAWp8NQ%3D&reserved=0
>>>
>>>> is
>>>>>>>> originally intended as a reply to this thread. Sorry for the
>>>>>>>> confusion.
>>>>>>>>
>>>>>>>> Kengo Seki <sekikn@apache.org <ma...@apache.org>>
>>>>>>>>
>>>>>>>> On Mon, Jun 20, 2022 at 11:31 AM Kengo Seki <sekikn@apache.org <
>>>> ma...@apache.org>> wrote:
>>>>>>>>>
>>>>>>>>> Thank you so much for writing the draft and commenting on it,
>>>> Brahma and Viraj!
>>>>>>>>> Firstly, a minor comment:
>>>>>>>>>
>>>>>>>>>> Trunk Code : Parallel we can work for the trunk code, this we
>>>> might give as 2.8.
>>>>>>>>>> * Working on bigtop mpack
>>>>>>>>>
>>>>>>>>> This means adding a new built-in stack which uses the Bigtop
>>>>>>>>> repository, and not developing Mpack on the Bigtop side, right?
>>>>>>>>> If so, "bigtop mpack" sounds confusing a bit.
>>>>>>>>>
>>>>>>>>>> hdp-select can stay on branch-2.7 and it can be moved to
>>>> bigtop for trunk branch. Thoughts?
>>>>>>>>>
>>>>>>>>> Sounds good to me. During the 2.7.x releases, users can use the
>>>> HDP
>>>>>>>>> stack by default. In addition, they can also add the Bigtop
>>>> stack
>>>>>>>>> through the Mpack, as Zhiguo mentioned in [1].
>>>>>>>>> Since 2.8 or 3.0, we will drop HDP and users can use the
>>>> built-in
>>>>>>>>> Bigtop stack by default. It also allows us to drop the Mpack on
>>>> the
>>>>>>>>> Bigtop side.
>>>>>>>>>
>>>>>>>>> Regarding the built-in Bigtop stack developed on trunk, the
>>>> current
>>>>>>>>> BIGTOP stack in Ambari [2] may be a good start point to be
>>>> upgraded.
>>>>>>>>> I ensured that Ambari 2.7.6 was successfully built with the
>>>>>>>>> `-Dstack.distribution=BIGTOP` option, though Bigtop 0.8 is too
>>>> old to
>>>>>>>>> use for now.
>>>>>>>>>
>>>>>>>>> [1]:
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iqQ6QCM1%2BMYJ61tU%2FFk%2BWC8GvftYYGhrnu6V%2FFJ3d7o%3D&reserved=0
>>>> <
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iqQ6QCM1%2BMYJ61tU%2FFk%2BWC8GvftYYGhrnu6V%2FFJ3d7o%3D&reserved=0
>>>>>
>>>>>>>>> [2]:
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=FR5NTHCZt%2FQcqy9NyT63Q3NIYlEeJ6sIpbOqzfDOMRY%3D&reserved=0
>>>> <
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
>>>> [message truncated...]
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
>>>> For additional commands, e-mail: dev-help@ambari.apache.org
>>>>
>>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
>> For additional commands, e-mail: dev-help@ambari.apache.org
>>
>>
>
Re: [Discuss] Initial Draft for Roadmap
Posted by 吴治国 <wz...@163.com>.
Hi 哲远,
Thanks for your suggestion, I'm also considering add those components in the future.
But now we should focus on next release.
BTW, ClickHouse is under discussion, I’ll start working on it soon or later if we all agree to include it.
Best Regards,
Zhiguo Wu
> On Aug 22, 2022, at 11:31, 王哲远 <wa...@gmail.com> wrote:
>
> I think we should add the data lake(Delta Lake, Hudi or Iceberg) to the
> stack for the next release and OLAP(ClickHouse or Doris) for
> future releases.
>
> 吴治国 <wz...@163.com> 于2022年8月17日周三 15:20写道：
>
>> Hi Jun,
>>
>> Sorry I miss some details here.
>>
>> After we move Bigtop Ambari MPack to Ambari side, it's no longer a
>> management-pack[1] for Ambari.
>> It'll be the default stack[2] which will replace HDP while compiling[3],
>> also, we can remove MPack code on Bigtop side[4], you can find old version
>> of bigtop stack in Ambari release 2.7.6[5].
>>
>> The reason I bring up Frontend Refactor is that the framework is outdated,
>> which cause it difficult to add new features, it also has compiled issue in
>> China, but the work won't be done in release 2.8.0, we can discuss in the
>> future.
>>
>> In addition, the development of components provided by Bigtop 3.2.0 is
>> almost done[6], I think we only need three more components(Flink, Spark3
>> and Phoenix, maybe Alluxio also needed?) to prepare the next release, but
>> there are some code still need to be updated after Bigtop provide the
>> packages with prefix in the installation path(which is important for stack
>> upgrade).
>>
>> Best Regards,
>> Zhiguo Wu
>>
>> [1]: https://github.com/apache/ambari/tree/trunk/contrib/management-packs
>> [2]:
>> https://github.com/apache/ambari/tree/trunk/ambari-server/src/main/resources/stacks
>> [3]: https://github.com/apache/ambari/blob/trunk/pom.xml#L90
>> [4]:
>> https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/bigtop-ambari-mpack
>> [5]:
>> https://github.com/apache/ambari/tree/release-2.7.6/ambari-server/src/main/resources/stacks
>> [6]:
>> https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/bigtop-ambari-mpack/bgtp-ambari-mpack/src/main/resources/stacks/BGTP/1.0/services
>>
>>> On Aug 17, 2022, at 11:08, Jun HE <ju...@apache.org> wrote:
>>>
>>> I agree with "1. Fix some security problems by upgrading Spring Framework
>>> and others".Security is important and we can use this as a ramp-up as the
>>> community is gradually bringing everything in Ambari back on track, like
>> PR
>>> review, CI jobs,...
>>>
>>> For other things like "frontend refactor", it would be good to have more
>>> inputs on existing issues/limitations before we decide to "refactor".
>>> For the mpack part, my understanding is that "management packs" is a
>>> framework that allows different stack vendors to provide a compatible
>> stack
>>> that can be maintained and operated by Ambari. There could be minor
>>> differences between different vendors. So the definition like
>>> path/repo/name etc should be the vendor's decision instead of being
>> defined
>>> by Ambari. I would suggest leaving the mpack related work to a certain
>>> community, for example, Bigtop, and Ambari just review and accept their
>> PR
>>> just for the user's convenience.
>>>
>>> Just my 2 cents. And happy to learn any comments.
>>>
>>> 吴治国 <wz...@163.com> 于2022年8月16日周二 17:50写道：
>>>
>>>> Hi community,
>>>>
>>>> In my original idea, I was planning to provide Bigtop3.1.0 + Ambari2.7.5
>>>> for users at first, and keep all big changes in Ambari2.8.0, e.g.
>>>> 1、Fix some security problems by upgrading Spring Framework and others
>>>> 2、Frontend refactor
>>>> 3、Upgrade python version
>>>> etc.
>>>>
>>>> All these changes will take long time, which may be bad for the release
>>>> cycle of Ambari itself.
>>>> So I'm considering we can move Mpack development to Ambari side, and
>> 2.8.0
>>>> is all about Bigtop stack, move other big changes to next major
>>>> version(except the security problem, which absolutely need to be fixed
>> in
>>>> next release).
>>>>
>>>> But we need to make final decision for the things we discussed before,
>>>> like,
>>>> 1、Add prefix to installation path(/usr/bigtop)
>>>> 2、Which repo to store the packages(with prefix in installation path)?
>>>> 3、Related name(Stack name: BIGTOP, scripts: conf-select, distro-select,
>>>> bigtop-select are all ok for me)?
>>>> Any other things I forgot?
>>>>
>>>> What’s your opinion?
>>>>
>>>> Best Regards,
>>>> Zhiguo Wu
>>>>
>>>> On 2022/06/23 06:26:44 "Battula, Brahma Reddy" wrote:
>>>>>
>>>>> IMO, First way is not heavier when we consider all the efforts for
>> mpack
>>>> with upgrades. And first one is easiest to adopt now.
>>>>>
>>>>> We'll start discussing the bigtop mailing list further.
>>>>>
>>>>>
>>>>> On 21/06/22, 1:11 PM, "吴治国" <wz...@163.com> wrote:
>>>>>
>>>>> Thanks for the details, Brahma and Kengo.
>>>>>
>>>>> If I understand it correctly, there are two approaches under
>>>> discussion:
>>>>>
>>>>> 1、Prepare to release 2.7.7, in this version, we won't use HDP
>>>> components anymore, we'll replace it with the community version
>> provided by
>>>> Bigtop, and update RPM and DEB build scripts provided by Bigtop, change
>> the
>>>> installation path so hdp-select can be reused. (Due to component version
>>>> changes, I don't know if the code under /stacks needs to be updated)
>>>>> 2、Develop Bigtop mpack for Ambari 2.7.5, we don't need to change
>>>> Ambari's code (if the code under /stacks can be reused in the first way,
>>>> maybe it also works in this way?)
>>>>>
>>>>> Isn't the workload of the first way heavier?
>>>>>
>>>>>
>>>>> About the meeting
>>>>>
>>>>> I thought maybe email discussion would be better?
>>>>> The discussion can be open for a few days and all interested people
>>>> in the community can participate, not just a few of us.
>>>>> And due to time zone and language issues, many people are hard to
>>>> attending the meetings, email discussions may be more suitable for these
>>>> people.
>>>>>
>>>>> What do you think?
>>>>>
>>>>> Best Regards,
>>>>> Zhiguo Wu
>>>>>
>>>>>> On Jun 20, 2022, at 23:15, Battula, Brahma Reddy
>>>> <bb...@visa.com.INVALID> wrote:
>>>>>>
>>>>>>>> Assuming the understanding above is correct, my new concerns are:
>>>>>>
>>>>>> Yes, Hope we'll in same page now. Still we can discuss in weekly
>>>> call.
>>>>>>
>>>>>>>> * Can we still call it as "HDP"? Doesn't it infringe Cloudera's
>>>> trademark?
>>>>>>>> If so, should we give a new stack name to it, as you mentioned as
>>>>>>>> planA in [3]?
>>>>>>>> (or it makes upgrading HDP cluster difficult? I don't understand
>>>> it so much)
>>>>>>
>>>>>> Yes, it may be. Hence I introduced the planA and preferred it. one
>>>> more idea(bad) came in my mind that can we treat "HDP: Hadoop Data
>>>> Platform", not sure whether it should be ok or not(and we might not
>>>> delivering as distro's).
>>>>>>
>>>>>>>> * It may be a bit tough work to fix Bigtop's build scripts so
>>>> that its
>>>>>>>> artifacts match Ambari's requirements.
>>>>>>>> (package versioning, file layouts, included and not included
>>>> files,
>>>>>>>> file permissions, user/group creation, etc.)
>>>>>>
>>>>>> Only paths matter right everything else can be re-used from the
>>>> hdp-select.
>>>>>>
>>>>>>
>>>>>> On 20/06/22, 8:13 PM, "Kengo Seki" <sekikn@apache.org <
>>>> ma...@apache.org>> wrote:
>>>>>>
>>>>>> Thanks for the elaboration Brahma, but I'm a bit confused.
>>>>>> At this point, my understanding is as follows. Are these correct?
>>>>>>
>>>>>> * In the 2.7.x line, we'll still use a stack definition based on
>>>> HDP's one [1].
>>>>>>
>>>>>> * But regarding RPM and DEB packages, we'll build them from source
>>>> by leveraging
>>>>>> Bigtop's build scripts and publish them on the ASF distribution
>>>> site
>>>>>> or somewhere.
>>>>>> repoinfo.xml [2] will also be fixed to point that repository.
>>>>>>
>>>>>> Assuming the understanding above is correct, my new concerns are:
>>>>>>
>>>>>> * Can we still call it as "HDP"? Doesn't it infringe Cloudera's
>>>> trademark?
>>>>>> If so, should we give a new stack name to it, as you mentioned as
>>>>>> planA in [3]?
>>>>>> (or it makes upgrading HDP cluster difficult? I don't understand
>>>> it so much)
>>>>>>
>>>>>> * It may be a bit tough work to fix Bigtop's build scripts so that
>>>> its
>>>>>> artifacts match Ambari's requirements.
>>>>>> (package versioning, file layouts, included and not included files,
>>>>>> file permissions, user/group creation, etc.)
>>>>>> I'm not sure if it's faster than developing Mpack.
>>>>>>
>>>>>> [1]:
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dFX07wfIY8aukfEUcJe9GN0rds8t3%2Fpenmsiszglkn4%3D&reserved=0
>>>> <
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dFX07wfIY8aukfEUcJe9GN0rds8t3%2Fpenmsiszglkn4%3D&reserved=0
>>>>>
>>>>>> [2]:
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP%2F2.6%2Frepos%2Frepoinfo.xml&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MDz57IloHfWYWcwmAI9h1pYDG45v1UbU6K2M%2BkpcDEA%3D&reserved=0
>>>> <
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP%2F2.6%2Frepos%2Frepoinfo.xml&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MDz57IloHfWYWcwmAI9h1pYDG45v1UbU6K2M%2BkpcDEA%3D&reserved=0
>>>>>
>>>>>> [3]:
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm77lhzo0njr2dhoock5twtm0c19j2py8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=P1pPHwM1YggdPT9qD4aAvJdehjjZdZ8mLhvn0ZfR7Gw%3D&reserved=0
>>>> <
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm77lhzo0njr2dhoock5twtm0c19j2py8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=P1pPHwM1YggdPT9qD4aAvJdehjjZdZ8mLhvn0ZfR7Gw%3D&reserved=0
>>>>>
>>>>>>
>>>>>> Kengo Seki <sekikn@gmail.com <ma...@gmail.com>>
>>>>>>
>>>>>> On Mon, Jun 20, 2022 at 8:05 PM Battula, Brahma Reddy
>>>>>> <bbattula@visa.com.invalid <ma...@visa.com.invalid>> wrote:
>>>>>>>
>>>>>>> We'll not use the HDP Packages (Only selects we'll use from hdp,
>>>> which can be plugged in bigtop). Still we'll use opensource packages and
>>>> bigtop to create the RPM's and selects.
>>>>>>> Just we can use the source tar balls opensource components(e.g
>>>> Hadoop-3.2.3 source tarballs from apache repo) and build rpm's and
>> publish
>>>> in following repo. Or from the apache github repo's.
>>>>>>>
>>>>>>>
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2Fbigtop%2Fbigtop-3.1.0%2Frepos%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RyNyy7fOUlWBSFMJdd1iRMrmlKOVyA5NEGuMpSkCSEs%3D&reserved=0
>>>> <
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2Fbigtop%2Fbigtop-3.1.0%2Frepos%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RyNyy7fOUlWBSFMJdd1iRMrmlKOVyA5NEGuMpSkCSEs%3D&reserved=0
>>>>>
>>>>>>>
>>>>>>> Planning to have sync up call on this week, once it's finalize,
>>>> we can summarise here the approach. What do you think..?
>>>>>>>
>>>>>>>
>>>>>>> On 20/06/22, 4:06 PM, "Kengo Seki" <sekikn@apache.org <
>>>> ma...@apache.org>> wrote:
>>>>>>>
>>>>>>> Thanks for the reply, Brahma!
>>>>>>>
>>>>>>>> We'll use following repo, how other components in Apache are
>>>> hosted.
>>>>>>>
>>>>>>> Using that site for distribution has no problem. My concern is,
>>>> do we
>>>>>>> have the source tree for building HDP packages and does it still
>>>> work
>>>>>>> without the artifacts behind the paywall?
>>>>>>> According to ASF's release policy [1], we must publish the
>>>>>>> corresponding source releases whenever we publish binary
>>>> artifacts,
>>>>>>> such as RPM and DEB packages for HDP.
>>>>>>>
>>>>>>> [1]:
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Flegal%2Frelease-policy.html%23compiled-packages&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qPEbEEpIN8NCsrMnxW5xKbB%2B19bDJprq%2FiXjIF09vls%3D&reserved=0
>>>> <
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Flegal%2Frelease-policy.html%23compiled-packages&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qPEbEEpIN8NCsrMnxW5xKbB%2B19bDJprq%2FiXjIF09vls%3D&reserved=0
>>>>>
>>>>>>>
>>>>>>> Kengo Seki <sekikn@apache.org <ma...@apache.org>>
>>>>>>>
>>>>>>> On Mon, Jun 20, 2022 at 6:19 PM Battula, Brahma Reddy
>>>>>>> <bbattula@visa.com.invalid <ma...@visa.com.invalid>> wrote:
>>>>>>>>
>>>>>>>> Hi Kengo Seki,
>>>>>>>>
>>>>>>>> Thanks for pitching here.. We'll use following repo, how other
>>>> components in Apache are hosted. Any other suggestions..?
>>>>>>>>
>>>>>>>>
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OUGhE45KsDvPum4%2Fd9cwlJPfH66OlKE%2BgD557%2BiiAC8%3D&reserved=0
>>>> <
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OUGhE45KsDvPum4%2Fd9cwlJPfH66OlKE%2BgD557%2BiiAC8%3D&reserved=0
>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 20/06/22, 2:32 PM, "Kengo Seki" <sekikn@apache.org <
>>>> ma...@apache.org>> wrote:
>>>>>>>>
>>>>>>>> Oh, I've just noticed that I submitted an additional question
>>>> into the
>>>>>>>> wrong thread mistakenly.
>>>>>>>>
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fj6kc2h1lphvjvsblcthcl5yzzmjk3tb5&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UTDhFu1Bloa3FzD0shWWHoDt%2FrQD22JziV1%2BNAWp8NQ%3D&reserved=0
>>>> <
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fj6kc2h1lphvjvsblcthcl5yzzmjk3tb5&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UTDhFu1Bloa3FzD0shWWHoDt%2FrQD22JziV1%2BNAWp8NQ%3D&reserved=0
>>>
>>>> is
>>>>>>>> originally intended as a reply to this thread. Sorry for the
>>>>>>>> confusion.
>>>>>>>>
>>>>>>>> Kengo Seki <sekikn@apache.org <ma...@apache.org>>
>>>>>>>>
>>>>>>>> On Mon, Jun 20, 2022 at 11:31 AM Kengo Seki <sekikn@apache.org <
>>>> ma...@apache.org>> wrote:
>>>>>>>>>
>>>>>>>>> Thank you so much for writing the draft and commenting on it,
>>>> Brahma and Viraj!
>>>>>>>>> Firstly, a minor comment:
>>>>>>>>>
>>>>>>>>>> Trunk Code : Parallel we can work for the trunk code, this we
>>>> might give as 2.8.
>>>>>>>>>> * Working on bigtop mpack
>>>>>>>>>
>>>>>>>>> This means adding a new built-in stack which uses the Bigtop
>>>>>>>>> repository, and not developing Mpack on the Bigtop side, right?
>>>>>>>>> If so, "bigtop mpack" sounds confusing a bit.
>>>>>>>>>
>>>>>>>>>> hdp-select can stay on branch-2.7 and it can be moved to
>>>> bigtop for trunk branch. Thoughts?
>>>>>>>>>
>>>>>>>>> Sounds good to me. During the 2.7.x releases, users can use the
>>>> HDP
>>>>>>>>> stack by default. In addition, they can also add the Bigtop
>>>> stack
>>>>>>>>> through the Mpack, as Zhiguo mentioned in [1].
>>>>>>>>> Since 2.8 or 3.0, we will drop HDP and users can use the
>>>> built-in
>>>>>>>>> Bigtop stack by default. It also allows us to drop the Mpack on
>>>> the
>>>>>>>>> Bigtop side.
>>>>>>>>>
>>>>>>>>> Regarding the built-in Bigtop stack developed on trunk, the
>>>> current
>>>>>>>>> BIGTOP stack in Ambari [2] may be a good start point to be
>>>> upgraded.
>>>>>>>>> I ensured that Ambari 2.7.6 was successfully built with the
>>>>>>>>> `-Dstack.distribution=BIGTOP` option, though Bigtop 0.8 is too
>>>> old to
>>>>>>>>> use for now.
>>>>>>>>>
>>>>>>>>> [1]:
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iqQ6QCM1%2BMYJ61tU%2FFk%2BWC8GvftYYGhrnu6V%2FFJ3d7o%3D&reserved=0
>>>> <
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iqQ6QCM1%2BMYJ61tU%2FFk%2BWC8GvftYYGhrnu6V%2FFJ3d7o%3D&reserved=0
>>>>>
>>>>>>>>> [2]:
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=FR5NTHCZt%2FQcqy9NyT63Q3NIYlEeJ6sIpbOqzfDOMRY%3D&reserved=0
>>>> <
>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
>>>> [message truncated...]
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
>>>> For additional commands, e-mail: dev-help@ambari.apache.org
>>>>
>>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
>> For additional commands, e-mail: dev-help@ambari.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For additional commands, e-mail: dev-help@ambari.apache.org
Re: [Discuss] Initial Draft for Roadmap
Posted by 王哲远 <wa...@gmail.com>.
I think we should add the data lake(Delta Lake, Hudi or Iceberg) to the
stack for the next release and OLAP(ClickHouse or Doris) for
future releases.
吴治国 <wz...@163.com> 于2022年8月17日周三 15:20写道：
> Hi Jun,
>
> Sorry I miss some details here.
>
> After we move Bigtop Ambari MPack to Ambari side, it's no longer a
> management-pack[1] for Ambari.
> It'll be the default stack[2] which will replace HDP while compiling[3],
> also, we can remove MPack code on Bigtop side[4], you can find old version
> of bigtop stack in Ambari release 2.7.6[5].
>
> The reason I bring up Frontend Refactor is that the framework is outdated,
> which cause it difficult to add new features, it also has compiled issue in
> China, but the work won't be done in release 2.8.0, we can discuss in the
> future.
>
> In addition, the development of components provided by Bigtop 3.2.0 is
> almost done[6], I think we only need three more components(Flink, Spark3
> and Phoenix, maybe Alluxio also needed?) to prepare the next release, but
> there are some code still need to be updated after Bigtop provide the
> packages with prefix in the installation path(which is important for stack
> upgrade).
>
> Best Regards,
> Zhiguo Wu
>
> [1]: https://github.com/apache/ambari/tree/trunk/contrib/management-packs
> [2]:
> https://github.com/apache/ambari/tree/trunk/ambari-server/src/main/resources/stacks
> [3]: https://github.com/apache/ambari/blob/trunk/pom.xml#L90
> [4]:
> https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/bigtop-ambari-mpack
> [5]:
> https://github.com/apache/ambari/tree/release-2.7.6/ambari-server/src/main/resources/stacks
> [6]:
> https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/bigtop-ambari-mpack/bgtp-ambari-mpack/src/main/resources/stacks/BGTP/1.0/services
>
> > On Aug 17, 2022, at 11:08, Jun HE <ju...@apache.org> wrote:
> >
> > I agree with "1. Fix some security problems by upgrading Spring Framework
> > and others".Security is important and we can use this as a ramp-up as the
> > community is gradually bringing everything in Ambari back on track, like
> PR
> > review, CI jobs,...
> >
> > For other things like "frontend refactor", it would be good to have more
> > inputs on existing issues/limitations before we decide to "refactor".
> > For the mpack part, my understanding is that "management packs" is a
> > framework that allows different stack vendors to provide a compatible
> stack
> > that can be maintained and operated by Ambari. There could be minor
> > differences between different vendors. So the definition like
> > path/repo/name etc should be the vendor's decision instead of being
> defined
> > by Ambari. I would suggest leaving the mpack related work to a certain
> > community, for example, Bigtop, and Ambari just review and accept their
> PR
> > just for the user's convenience.
> >
> > Just my 2 cents. And happy to learn any comments.
> >
> > 吴治国 <wz...@163.com> 于2022年8月16日周二 17:50写道：
> >
> >> Hi community,
> >>
> >> In my original idea, I was planning to provide Bigtop3.1.0 + Ambari2.7.5
> >> for users at first, and keep all big changes in Ambari2.8.0, e.g.
> >> 1、Fix some security problems by upgrading Spring Framework and others
> >> 2、Frontend refactor
> >> 3、Upgrade python version
> >> etc.
> >>
> >> All these changes will take long time, which may be bad for the release
> >> cycle of Ambari itself.
> >> So I'm considering we can move Mpack development to Ambari side, and
> 2.8.0
> >> is all about Bigtop stack, move other big changes to next major
> >> version(except the security problem, which absolutely need to be fixed
> in
> >> next release).
> >>
> >> But we need to make final decision for the things we discussed before,
> >> like,
> >> 1、Add prefix to installation path(/usr/bigtop)
> >> 2、Which repo to store the packages(with prefix in installation path)?
> >> 3、Related name(Stack name: BIGTOP, scripts: conf-select, distro-select,
> >> bigtop-select are all ok for me)?
> >> Any other things I forgot?
> >>
> >> What’s your opinion?
> >>
> >> Best Regards,
> >> Zhiguo Wu
> >>
> >> On 2022/06/23 06:26:44 "Battula, Brahma Reddy" wrote:
> >>>
> >>> IMO, First way is not heavier when we consider all the efforts for
> mpack
> >> with upgrades. And first one is easiest to adopt now.
> >>>
> >>> We'll start discussing the bigtop mailing list further.
> >>>
> >>>
> >>> On 21/06/22, 1:11 PM, "吴治国" <wz...@163.com> wrote:
> >>>
> >>> Thanks for the details, Brahma and Kengo.
> >>>
> >>> If I understand it correctly, there are two approaches under
> >> discussion:
> >>>
> >>> 1、Prepare to release 2.7.7, in this version, we won't use HDP
> >> components anymore, we'll replace it with the community version
> provided by
> >> Bigtop, and update RPM and DEB build scripts provided by Bigtop, change
> the
> >> installation path so hdp-select can be reused. (Due to component version
> >> changes, I don't know if the code under /stacks needs to be updated)
> >>> 2、Develop Bigtop mpack for Ambari 2.7.5, we don't need to change
> >> Ambari's code (if the code under /stacks can be reused in the first way,
> >> maybe it also works in this way?)
> >>>
> >>> Isn't the workload of the first way heavier?
> >>>
> >>>
> >>> About the meeting
> >>>
> >>> I thought maybe email discussion would be better?
> >>> The discussion can be open for a few days and all interested people
> >> in the community can participate, not just a few of us.
> >>> And due to time zone and language issues, many people are hard to
> >> attending the meetings, email discussions may be more suitable for these
> >> people.
> >>>
> >>> What do you think?
> >>>
> >>> Best Regards,
> >>> Zhiguo Wu
> >>>
> >>>> On Jun 20, 2022, at 23:15, Battula, Brahma Reddy
> >> <bb...@visa.com.INVALID> wrote:
> >>>>
> >>>>>> Assuming the understanding above is correct, my new concerns are:
> >>>>
> >>>> Yes, Hope we'll in same page now. Still we can discuss in weekly
> >> call.
> >>>>
> >>>>>> * Can we still call it as "HDP"? Doesn't it infringe Cloudera's
> >> trademark?
> >>>>>> If so, should we give a new stack name to it, as you mentioned as
> >>>>>> planA in [3]?
> >>>>>> (or it makes upgrading HDP cluster difficult? I don't understand
> >> it so much)
> >>>>
> >>>> Yes, it may be. Hence I introduced the planA and preferred it. one
> >> more idea(bad) came in my mind that can we treat "HDP: Hadoop Data
> >> Platform", not sure whether it should be ok or not(and we might not
> >> delivering as distro's).
> >>>>
> >>>>>> * It may be a bit tough work to fix Bigtop's build scripts so
> >> that its
> >>>>>> artifacts match Ambari's requirements.
> >>>>>> (package versioning, file layouts, included and not included
> >> files,
> >>>>>> file permissions, user/group creation, etc.)
> >>>>
> >>>> Only paths matter right everything else can be re-used from the
> >> hdp-select.
> >>>>
> >>>>
> >>>> On 20/06/22, 8:13 PM, "Kengo Seki" <sekikn@apache.org <
> >> ma...@apache.org>> wrote:
> >>>>
> >>>> Thanks for the elaboration Brahma, but I'm a bit confused.
> >>>> At this point, my understanding is as follows. Are these correct?
> >>>>
> >>>> * In the 2.7.x line, we'll still use a stack definition based on
> >> HDP's one [1].
> >>>>
> >>>> * But regarding RPM and DEB packages, we'll build them from source
> >> by leveraging
> >>>> Bigtop's build scripts and publish them on the ASF distribution
> >> site
> >>>> or somewhere.
> >>>> repoinfo.xml [2] will also be fixed to point that repository.
> >>>>
> >>>> Assuming the understanding above is correct, my new concerns are:
> >>>>
> >>>> * Can we still call it as "HDP"? Doesn't it infringe Cloudera's
> >> trademark?
> >>>> If so, should we give a new stack name to it, as you mentioned as
> >>>> planA in [3]?
> >>>> (or it makes upgrading HDP cluster difficult? I don't understand
> >> it so much)
> >>>>
> >>>> * It may be a bit tough work to fix Bigtop's build scripts so that
> >> its
> >>>> artifacts match Ambari's requirements.
> >>>> (package versioning, file layouts, included and not included files,
> >>>> file permissions, user/group creation, etc.)
> >>>> I'm not sure if it's faster than developing Mpack.
> >>>>
> >>>> [1]:
> >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dFX07wfIY8aukfEUcJe9GN0rds8t3%2Fpenmsiszglkn4%3D&reserved=0
> >> <
> >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dFX07wfIY8aukfEUcJe9GN0rds8t3%2Fpenmsiszglkn4%3D&reserved=0
> >>>
> >>>> [2]:
> >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP%2F2.6%2Frepos%2Frepoinfo.xml&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MDz57IloHfWYWcwmAI9h1pYDG45v1UbU6K2M%2BkpcDEA%3D&reserved=0
> >> <
> >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP%2F2.6%2Frepos%2Frepoinfo.xml&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MDz57IloHfWYWcwmAI9h1pYDG45v1UbU6K2M%2BkpcDEA%3D&reserved=0
> >>>
> >>>> [3]:
> >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm77lhzo0njr2dhoock5twtm0c19j2py8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=P1pPHwM1YggdPT9qD4aAvJdehjjZdZ8mLhvn0ZfR7Gw%3D&reserved=0
> >> <
> >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm77lhzo0njr2dhoock5twtm0c19j2py8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=P1pPHwM1YggdPT9qD4aAvJdehjjZdZ8mLhvn0ZfR7Gw%3D&reserved=0
> >>>
> >>>>
> >>>> Kengo Seki <sekikn@gmail.com <ma...@gmail.com>>
> >>>>
> >>>> On Mon, Jun 20, 2022 at 8:05 PM Battula, Brahma Reddy
> >>>> <bbattula@visa.com.invalid <ma...@visa.com.invalid>> wrote:
> >>>>>
> >>>>> We'll not use the HDP Packages (Only selects we'll use from hdp,
> >> which can be plugged in bigtop). Still we'll use opensource packages and
> >> bigtop to create the RPM's and selects.
> >>>>> Just we can use the source tar balls opensource components(e.g
> >> Hadoop-3.2.3 source tarballs from apache repo) and build rpm's and
> publish
> >> in following repo. Or from the apache github repo's.
> >>>>>
> >>>>>
> >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2Fbigtop%2Fbigtop-3.1.0%2Frepos%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RyNyy7fOUlWBSFMJdd1iRMrmlKOVyA5NEGuMpSkCSEs%3D&reserved=0
> >> <
> >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2Fbigtop%2Fbigtop-3.1.0%2Frepos%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RyNyy7fOUlWBSFMJdd1iRMrmlKOVyA5NEGuMpSkCSEs%3D&reserved=0
> >>>
> >>>>>
> >>>>> Planning to have sync up call on this week, once it's finalize,
> >> we can summarise here the approach. What do you think..?
> >>>>>
> >>>>>
> >>>>> On 20/06/22, 4:06 PM, "Kengo Seki" <sekikn@apache.org <
> >> ma...@apache.org>> wrote:
> >>>>>
> >>>>> Thanks for the reply, Brahma!
> >>>>>
> >>>>>> We'll use following repo, how other components in Apache are
> >> hosted.
> >>>>>
> >>>>> Using that site for distribution has no problem. My concern is,
> >> do we
> >>>>> have the source tree for building HDP packages and does it still
> >> work
> >>>>> without the artifacts behind the paywall?
> >>>>> According to ASF's release policy [1], we must publish the
> >>>>> corresponding source releases whenever we publish binary
> >> artifacts,
> >>>>> such as RPM and DEB packages for HDP.
> >>>>>
> >>>>> [1]:
> >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Flegal%2Frelease-policy.html%23compiled-packages&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qPEbEEpIN8NCsrMnxW5xKbB%2B19bDJprq%2FiXjIF09vls%3D&reserved=0
> >> <
> >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Flegal%2Frelease-policy.html%23compiled-packages&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qPEbEEpIN8NCsrMnxW5xKbB%2B19bDJprq%2FiXjIF09vls%3D&reserved=0
> >>>
> >>>>>
> >>>>> Kengo Seki <sekikn@apache.org <ma...@apache.org>>
> >>>>>
> >>>>> On Mon, Jun 20, 2022 at 6:19 PM Battula, Brahma Reddy
> >>>>> <bbattula@visa.com.invalid <ma...@visa.com.invalid>> wrote:
> >>>>>>
> >>>>>> Hi Kengo Seki,
> >>>>>>
> >>>>>> Thanks for pitching here.. We'll use following repo, how other
> >> components in Apache are hosted. Any other suggestions..?
> >>>>>>
> >>>>>>
> >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OUGhE45KsDvPum4%2Fd9cwlJPfH66OlKE%2BgD557%2BiiAC8%3D&reserved=0
> >> <
> >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OUGhE45KsDvPum4%2Fd9cwlJPfH66OlKE%2BgD557%2BiiAC8%3D&reserved=0
> >>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On 20/06/22, 2:32 PM, "Kengo Seki" <sekikn@apache.org <
> >> ma...@apache.org>> wrote:
> >>>>>>
> >>>>>> Oh, I've just noticed that I submitted an additional question
> >> into the
> >>>>>> wrong thread mistakenly.
> >>>>>>
> >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fj6kc2h1lphvjvsblcthcl5yzzmjk3tb5&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UTDhFu1Bloa3FzD0shWWHoDt%2FrQD22JziV1%2BNAWp8NQ%3D&reserved=0
> >> <
> >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fj6kc2h1lphvjvsblcthcl5yzzmjk3tb5&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UTDhFu1Bloa3FzD0shWWHoDt%2FrQD22JziV1%2BNAWp8NQ%3D&reserved=0
> >
> >> is
> >>>>>> originally intended as a reply to this thread. Sorry for the
> >>>>>> confusion.
> >>>>>>
> >>>>>> Kengo Seki <sekikn@apache.org <ma...@apache.org>>
> >>>>>>
> >>>>>> On Mon, Jun 20, 2022 at 11:31 AM Kengo Seki <sekikn@apache.org <
> >> ma...@apache.org>> wrote:
> >>>>>>>
> >>>>>>> Thank you so much for writing the draft and commenting on it,
> >> Brahma and Viraj!
> >>>>>>> Firstly, a minor comment:
> >>>>>>>
> >>>>>>>> Trunk Code : Parallel we can work for the trunk code, this we
> >> might give as 2.8.
> >>>>>>>> * Working on bigtop mpack
> >>>>>>>
> >>>>>>> This means adding a new built-in stack which uses the Bigtop
> >>>>>>> repository, and not developing Mpack on the Bigtop side, right?
> >>>>>>> If so, "bigtop mpack" sounds confusing a bit.
> >>>>>>>
> >>>>>>>> hdp-select can stay on branch-2.7 and it can be moved to
> >> bigtop for trunk branch. Thoughts?
> >>>>>>>
> >>>>>>> Sounds good to me. During the 2.7.x releases, users can use the
> >> HDP
> >>>>>>> stack by default. In addition, they can also add the Bigtop
> >> stack
> >>>>>>> through the Mpack, as Zhiguo mentioned in [1].
> >>>>>>> Since 2.8 or 3.0, we will drop HDP and users can use the
> >> built-in
> >>>>>>> Bigtop stack by default. It also allows us to drop the Mpack on
> >> the
> >>>>>>> Bigtop side.
> >>>>>>>
> >>>>>>> Regarding the built-in Bigtop stack developed on trunk, the
> >> current
> >>>>>>> BIGTOP stack in Ambari [2] may be a good start point to be
> >> upgraded.
> >>>>>>> I ensured that Ambari 2.7.6 was successfully built with the
> >>>>>>> `-Dstack.distribution=BIGTOP` option, though Bigtop 0.8 is too
> >> old to
> >>>>>>> use for now.
> >>>>>>>
> >>>>>>> [1]:
> >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iqQ6QCM1%2BMYJ61tU%2FFk%2BWC8GvftYYGhrnu6V%2FFJ3d7o%3D&reserved=0
> >> <
> >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iqQ6QCM1%2BMYJ61tU%2FFk%2BWC8GvftYYGhrnu6V%2FFJ3d7o%3D&reserved=0
> >>>
> >>>>>>> [2]:
> >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=FR5NTHCZt%2FQcqy9NyT63Q3NIYlEeJ6sIpbOqzfDOMRY%3D&reserved=0
> >> <
> >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
> >> [message truncated...]
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
> >> For additional commands, e-mail: dev-help@ambari.apache.org
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
> For additional commands, e-mail: dev-help@ambari.apache.org
>
>
Re: [Discuss] Initial Draft for Roadmap
Posted by 吴治国 <wz...@163.com>.
Hi Jun,
Sorry I miss some details here.
After we move Bigtop Ambari MPack to Ambari side, it's no longer a management-pack[1] for Ambari.
It'll be the default stack[2] which will replace HDP while compiling[3], also, we can remove MPack code on Bigtop side[4], you can find old version of bigtop stack in Ambari release 2.7.6[5].
The reason I bring up Frontend Refactor is that the framework is outdated, which cause it difficult to add new features, it also has compiled issue in China, but the work won't be done in release 2.8.0, we can discuss in the future.
In addition, the development of components provided by Bigtop 3.2.0 is almost done[6], I think we only need three more components(Flink, Spark3 and Phoenix, maybe Alluxio also needed?) to prepare the next release, but there are some code still need to be updated after Bigtop provide the packages with prefix in the installation path(which is important for stack upgrade).
Best Regards,
Zhiguo Wu
[1]: https://github.com/apache/ambari/tree/trunk/contrib/management-packs
[2]: https://github.com/apache/ambari/tree/trunk/ambari-server/src/main/resources/stacks
[3]: https://github.com/apache/ambari/blob/trunk/pom.xml#L90
[4]: https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/bigtop-ambari-mpack
[5]: https://github.com/apache/ambari/tree/release-2.7.6/ambari-server/src/main/resources/stacks
[6]: https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/bigtop-ambari-mpack/bgtp-ambari-mpack/src/main/resources/stacks/BGTP/1.0/services
> On Aug 17, 2022, at 11:08, Jun HE <ju...@apache.org> wrote:
>
> I agree with "1. Fix some security problems by upgrading Spring Framework
> and others".Security is important and we can use this as a ramp-up as the
> community is gradually bringing everything in Ambari back on track, like PR
> review, CI jobs,...
>
> For other things like "frontend refactor", it would be good to have more
> inputs on existing issues/limitations before we decide to "refactor".
> For the mpack part, my understanding is that "management packs" is a
> framework that allows different stack vendors to provide a compatible stack
> that can be maintained and operated by Ambari. There could be minor
> differences between different vendors. So the definition like
> path/repo/name etc should be the vendor's decision instead of being defined
> by Ambari. I would suggest leaving the mpack related work to a certain
> community, for example, Bigtop, and Ambari just review and accept their PR
> just for the user's convenience.
>
> Just my 2 cents. And happy to learn any comments.
>
> 吴治国 <wz...@163.com> 于2022年8月16日周二 17:50写道：
>
>> Hi community,
>>
>> In my original idea, I was planning to provide Bigtop3.1.0 + Ambari2.7.5
>> for users at first, and keep all big changes in Ambari2.8.0, e.g.
>> 1、Fix some security problems by upgrading Spring Framework and others
>> 2、Frontend refactor
>> 3、Upgrade python version
>> etc.
>>
>> All these changes will take long time, which may be bad for the release
>> cycle of Ambari itself.
>> So I'm considering we can move Mpack development to Ambari side, and 2.8.0
>> is all about Bigtop stack, move other big changes to next major
>> version(except the security problem, which absolutely need to be fixed in
>> next release).
>>
>> But we need to make final decision for the things we discussed before,
>> like,
>> 1、Add prefix to installation path(/usr/bigtop)
>> 2、Which repo to store the packages(with prefix in installation path)?
>> 3、Related name(Stack name: BIGTOP, scripts: conf-select, distro-select,
>> bigtop-select are all ok for me)?
>> Any other things I forgot?
>>
>> What’s your opinion?
>>
>> Best Regards,
>> Zhiguo Wu
>>
>> On 2022/06/23 06:26:44 "Battula, Brahma Reddy" wrote:
>>>
>>> IMO, First way is not heavier when we consider all the efforts for mpack
>> with upgrades. And first one is easiest to adopt now.
>>>
>>> We'll start discussing the bigtop mailing list further.
>>>
>>>
>>> On 21/06/22, 1:11 PM, "吴治国" <wz...@163.com> wrote:
>>>
>>> Thanks for the details, Brahma and Kengo.
>>>
>>> If I understand it correctly, there are two approaches under
>> discussion:
>>>
>>> 1、Prepare to release 2.7.7, in this version, we won't use HDP
>> components anymore, we'll replace it with the community version provided by
>> Bigtop, and update RPM and DEB build scripts provided by Bigtop, change the
>> installation path so hdp-select can be reused. (Due to component version
>> changes, I don't know if the code under /stacks needs to be updated)
>>> 2、Develop Bigtop mpack for Ambari 2.7.5, we don't need to change
>> Ambari's code (if the code under /stacks can be reused in the first way,
>> maybe it also works in this way?)
>>>
>>> Isn't the workload of the first way heavier?
>>>
>>>
>>> About the meeting
>>>
>>> I thought maybe email discussion would be better?
>>> The discussion can be open for a few days and all interested people
>> in the community can participate, not just a few of us.
>>> And due to time zone and language issues, many people are hard to
>> attending the meetings, email discussions may be more suitable for these
>> people.
>>>
>>> What do you think?
>>>
>>> Best Regards,
>>> Zhiguo Wu
>>>
>>>> On Jun 20, 2022, at 23:15, Battula, Brahma Reddy
>> <bb...@visa.com.INVALID> wrote:
>>>>
>>>>>> Assuming the understanding above is correct, my new concerns are:
>>>>
>>>> Yes, Hope we'll in same page now. Still we can discuss in weekly
>> call.
>>>>
>>>>>> * Can we still call it as "HDP"? Doesn't it infringe Cloudera's
>> trademark?
>>>>>> If so, should we give a new stack name to it, as you mentioned as
>>>>>> planA in [3]?
>>>>>> (or it makes upgrading HDP cluster difficult? I don't understand
>> it so much)
>>>>
>>>> Yes, it may be. Hence I introduced the planA and preferred it. one
>> more idea(bad) came in my mind that can we treat "HDP: Hadoop Data
>> Platform", not sure whether it should be ok or not(and we might not
>> delivering as distro's).
>>>>
>>>>>> * It may be a bit tough work to fix Bigtop's build scripts so
>> that its
>>>>>> artifacts match Ambari's requirements.
>>>>>> (package versioning, file layouts, included and not included
>> files,
>>>>>> file permissions, user/group creation, etc.)
>>>>
>>>> Only paths matter right everything else can be re-used from the
>> hdp-select.
>>>>
>>>>
>>>> On 20/06/22, 8:13 PM, "Kengo Seki" <sekikn@apache.org <
>> ma...@apache.org>> wrote:
>>>>
>>>> Thanks for the elaboration Brahma, but I'm a bit confused.
>>>> At this point, my understanding is as follows. Are these correct?
>>>>
>>>> * In the 2.7.x line, we'll still use a stack definition based on
>> HDP's one [1].
>>>>
>>>> * But regarding RPM and DEB packages, we'll build them from source
>> by leveraging
>>>> Bigtop's build scripts and publish them on the ASF distribution
>> site
>>>> or somewhere.
>>>> repoinfo.xml [2] will also be fixed to point that repository.
>>>>
>>>> Assuming the understanding above is correct, my new concerns are:
>>>>
>>>> * Can we still call it as "HDP"? Doesn't it infringe Cloudera's
>> trademark?
>>>> If so, should we give a new stack name to it, as you mentioned as
>>>> planA in [3]?
>>>> (or it makes upgrading HDP cluster difficult? I don't understand
>> it so much)
>>>>
>>>> * It may be a bit tough work to fix Bigtop's build scripts so that
>> its
>>>> artifacts match Ambari's requirements.
>>>> (package versioning, file layouts, included and not included files,
>>>> file permissions, user/group creation, etc.)
>>>> I'm not sure if it's faster than developing Mpack.
>>>>
>>>> [1]:
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dFX07wfIY8aukfEUcJe9GN0rds8t3%2Fpenmsiszglkn4%3D&reserved=0
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dFX07wfIY8aukfEUcJe9GN0rds8t3%2Fpenmsiszglkn4%3D&reserved=0
>>>
>>>> [2]:
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP%2F2.6%2Frepos%2Frepoinfo.xml&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MDz57IloHfWYWcwmAI9h1pYDG45v1UbU6K2M%2BkpcDEA%3D&reserved=0
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP%2F2.6%2Frepos%2Frepoinfo.xml&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MDz57IloHfWYWcwmAI9h1pYDG45v1UbU6K2M%2BkpcDEA%3D&reserved=0
>>>
>>>> [3]:
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm77lhzo0njr2dhoock5twtm0c19j2py8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=P1pPHwM1YggdPT9qD4aAvJdehjjZdZ8mLhvn0ZfR7Gw%3D&reserved=0
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm77lhzo0njr2dhoock5twtm0c19j2py8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=P1pPHwM1YggdPT9qD4aAvJdehjjZdZ8mLhvn0ZfR7Gw%3D&reserved=0
>>>
>>>>
>>>> Kengo Seki <sekikn@gmail.com <ma...@gmail.com>>
>>>>
>>>> On Mon, Jun 20, 2022 at 8:05 PM Battula, Brahma Reddy
>>>> <bbattula@visa.com.invalid <ma...@visa.com.invalid>> wrote:
>>>>>
>>>>> We'll not use the HDP Packages (Only selects we'll use from hdp,
>> which can be plugged in bigtop). Still we'll use opensource packages and
>> bigtop to create the RPM's and selects.
>>>>> Just we can use the source tar balls opensource components(e.g
>> Hadoop-3.2.3 source tarballs from apache repo) and build rpm's and publish
>> in following repo. Or from the apache github repo's.
>>>>>
>>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2Fbigtop%2Fbigtop-3.1.0%2Frepos%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RyNyy7fOUlWBSFMJdd1iRMrmlKOVyA5NEGuMpSkCSEs%3D&reserved=0
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2Fbigtop%2Fbigtop-3.1.0%2Frepos%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RyNyy7fOUlWBSFMJdd1iRMrmlKOVyA5NEGuMpSkCSEs%3D&reserved=0
>>>
>>>>>
>>>>> Planning to have sync up call on this week, once it's finalize,
>> we can summarise here the approach. What do you think..?
>>>>>
>>>>>
>>>>> On 20/06/22, 4:06 PM, "Kengo Seki" <sekikn@apache.org <
>> ma...@apache.org>> wrote:
>>>>>
>>>>> Thanks for the reply, Brahma!
>>>>>
>>>>>> We'll use following repo, how other components in Apache are
>> hosted.
>>>>>
>>>>> Using that site for distribution has no problem. My concern is,
>> do we
>>>>> have the source tree for building HDP packages and does it still
>> work
>>>>> without the artifacts behind the paywall?
>>>>> According to ASF's release policy [1], we must publish the
>>>>> corresponding source releases whenever we publish binary
>> artifacts,
>>>>> such as RPM and DEB packages for HDP.
>>>>>
>>>>> [1]:
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Flegal%2Frelease-policy.html%23compiled-packages&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qPEbEEpIN8NCsrMnxW5xKbB%2B19bDJprq%2FiXjIF09vls%3D&reserved=0
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Flegal%2Frelease-policy.html%23compiled-packages&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qPEbEEpIN8NCsrMnxW5xKbB%2B19bDJprq%2FiXjIF09vls%3D&reserved=0
>>>
>>>>>
>>>>> Kengo Seki <sekikn@apache.org <ma...@apache.org>>
>>>>>
>>>>> On Mon, Jun 20, 2022 at 6:19 PM Battula, Brahma Reddy
>>>>> <bbattula@visa.com.invalid <ma...@visa.com.invalid>> wrote:
>>>>>>
>>>>>> Hi Kengo Seki,
>>>>>>
>>>>>> Thanks for pitching here.. We'll use following repo, how other
>> components in Apache are hosted. Any other suggestions..?
>>>>>>
>>>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OUGhE45KsDvPum4%2Fd9cwlJPfH66OlKE%2BgD557%2BiiAC8%3D&reserved=0
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OUGhE45KsDvPum4%2Fd9cwlJPfH66OlKE%2BgD557%2BiiAC8%3D&reserved=0
>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 20/06/22, 2:32 PM, "Kengo Seki" <sekikn@apache.org <
>> ma...@apache.org>> wrote:
>>>>>>
>>>>>> Oh, I've just noticed that I submitted an additional question
>> into the
>>>>>> wrong thread mistakenly.
>>>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fj6kc2h1lphvjvsblcthcl5yzzmjk3tb5&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UTDhFu1Bloa3FzD0shWWHoDt%2FrQD22JziV1%2BNAWp8NQ%3D&reserved=0
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fj6kc2h1lphvjvsblcthcl5yzzmjk3tb5&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UTDhFu1Bloa3FzD0shWWHoDt%2FrQD22JziV1%2BNAWp8NQ%3D&reserved=0>
>> is
>>>>>> originally intended as a reply to this thread. Sorry for the
>>>>>> confusion.
>>>>>>
>>>>>> Kengo Seki <sekikn@apache.org <ma...@apache.org>>
>>>>>>
>>>>>> On Mon, Jun 20, 2022 at 11:31 AM Kengo Seki <sekikn@apache.org <
>> ma...@apache.org>> wrote:
>>>>>>>
>>>>>>> Thank you so much for writing the draft and commenting on it,
>> Brahma and Viraj!
>>>>>>> Firstly, a minor comment:
>>>>>>>
>>>>>>>> Trunk Code : Parallel we can work for the trunk code, this we
>> might give as 2.8.
>>>>>>>> * Working on bigtop mpack
>>>>>>>
>>>>>>> This means adding a new built-in stack which uses the Bigtop
>>>>>>> repository, and not developing Mpack on the Bigtop side, right?
>>>>>>> If so, "bigtop mpack" sounds confusing a bit.
>>>>>>>
>>>>>>>> hdp-select can stay on branch-2.7 and it can be moved to
>> bigtop for trunk branch. Thoughts?
>>>>>>>
>>>>>>> Sounds good to me. During the 2.7.x releases, users can use the
>> HDP
>>>>>>> stack by default. In addition, they can also add the Bigtop
>> stack
>>>>>>> through the Mpack, as Zhiguo mentioned in [1].
>>>>>>> Since 2.8 or 3.0, we will drop HDP and users can use the
>> built-in
>>>>>>> Bigtop stack by default. It also allows us to drop the Mpack on
>> the
>>>>>>> Bigtop side.
>>>>>>>
>>>>>>> Regarding the built-in Bigtop stack developed on trunk, the
>> current
>>>>>>> BIGTOP stack in Ambari [2] may be a good start point to be
>> upgraded.
>>>>>>> I ensured that Ambari 2.7.6 was successfully built with the
>>>>>>> `-Dstack.distribution=BIGTOP` option, though Bigtop 0.8 is too
>> old to
>>>>>>> use for now.
>>>>>>>
>>>>>>> [1]:
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iqQ6QCM1%2BMYJ61tU%2FFk%2BWC8GvftYYGhrnu6V%2FFJ3d7o%3D&reserved=0
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iqQ6QCM1%2BMYJ61tU%2FFk%2BWC8GvftYYGhrnu6V%2FFJ3d7o%3D&reserved=0
>>>
>>>>>>> [2]:
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=FR5NTHCZt%2FQcqy9NyT63Q3NIYlEeJ6sIpbOqzfDOMRY%3D&reserved=0
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
>> [message truncated...]
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
>> For additional commands, e-mail: dev-help@ambari.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For additional commands, e-mail: dev-help@ambari.apache.org
Re: [Discuss] Initial Draft for Roadmap
Posted by 吴治国 <wz...@163.com>.
Hi Jun,
Sorry I miss some details here.
After we move Bigtop Ambari MPack to Ambari side, it's no longer a management-pack[1] for Ambari.
It'll be the default stack[2] which will replace HDP while compiling[3], also, we can remove MPack code on Bigtop side[4], you can find old version of bigtop stack in Ambari release 2.7.6[5].
The reason I bring up Frontend Refactor is that the framework is outdated, which cause it difficult to add new features, it also has compiled issue in China, but the work won't be done in release 2.8.0, we can discuss in the future.
In addition, the development of components provided by Bigtop 3.2.0 is almost done[6], I think we only need three more components(Flink, Spark3 and Phoenix, maybe Alluxio also needed?) to prepare the next release, but there are some code still need to be updated after Bigtop provide the packages with prefix in the installation path(which is important for stack upgrade).
Best Regards,
Zhiguo Wu
[1]: https://github.com/apache/ambari/tree/trunk/contrib/management-packs
[2]: https://github.com/apache/ambari/tree/trunk/ambari-server/src/main/resources/stacks
[3]: https://github.com/apache/ambari/blob/trunk/pom.xml#L90
[4]: https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/bigtop-ambari-mpack
[5]: https://github.com/apache/ambari/tree/release-2.7.6/ambari-server/src/main/resources/stacks
[6]: https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/bigtop-ambari-mpack/bgtp-ambari-mpack/src/main/resources/stacks/BGTP/1.0/services
> On Aug 17, 2022, at 11:08, Jun HE <ju...@apache.org> wrote:
>
> I agree with "1. Fix some security problems by upgrading Spring Framework
> and others".Security is important and we can use this as a ramp-up as the
> community is gradually bringing everything in Ambari back on track, like PR
> review, CI jobs,...
>
> For other things like "frontend refactor", it would be good to have more
> inputs on existing issues/limitations before we decide to "refactor".
> For the mpack part, my understanding is that "management packs" is a
> framework that allows different stack vendors to provide a compatible stack
> that can be maintained and operated by Ambari. There could be minor
> differences between different vendors. So the definition like
> path/repo/name etc should be the vendor's decision instead of being defined
> by Ambari. I would suggest leaving the mpack related work to a certain
> community, for example, Bigtop, and Ambari just review and accept their PR
> just for the user's convenience.
>
> Just my 2 cents. And happy to learn any comments.
>
> 吴治国 <wz...@163.com> 于2022年8月16日周二 17:50写道：
>
>> Hi community,
>>
>> In my original idea, I was planning to provide Bigtop3.1.0 + Ambari2.7.5
>> for users at first, and keep all big changes in Ambari2.8.0, e.g.
>> 1、Fix some security problems by upgrading Spring Framework and others
>> 2、Frontend refactor
>> 3、Upgrade python version
>> etc.
>>
>> All these changes will take long time, which may be bad for the release
>> cycle of Ambari itself.
>> So I'm considering we can move Mpack development to Ambari side, and 2.8.0
>> is all about Bigtop stack, move other big changes to next major
>> version(except the security problem, which absolutely need to be fixed in
>> next release).
>>
>> But we need to make final decision for the things we discussed before,
>> like,
>> 1、Add prefix to installation path(/usr/bigtop)
>> 2、Which repo to store the packages(with prefix in installation path)?
>> 3、Related name(Stack name: BIGTOP, scripts: conf-select, distro-select,
>> bigtop-select are all ok for me)?
>> Any other things I forgot?
>>
>> What’s your opinion?
>>
>> Best Regards,
>> Zhiguo Wu
>>
>> On 2022/06/23 06:26:44 "Battula, Brahma Reddy" wrote:
>>>
>>> IMO, First way is not heavier when we consider all the efforts for mpack
>> with upgrades. And first one is easiest to adopt now.
>>>
>>> We'll start discussing the bigtop mailing list further.
>>>
>>>
>>> On 21/06/22, 1:11 PM, "吴治国" <wz...@163.com> wrote:
>>>
>>> Thanks for the details, Brahma and Kengo.
>>>
>>> If I understand it correctly, there are two approaches under
>> discussion:
>>>
>>> 1、Prepare to release 2.7.7, in this version, we won't use HDP
>> components anymore, we'll replace it with the community version provided by
>> Bigtop, and update RPM and DEB build scripts provided by Bigtop, change the
>> installation path so hdp-select can be reused. (Due to component version
>> changes, I don't know if the code under /stacks needs to be updated)
>>> 2、Develop Bigtop mpack for Ambari 2.7.5, we don't need to change
>> Ambari's code (if the code under /stacks can be reused in the first way,
>> maybe it also works in this way?)
>>>
>>> Isn't the workload of the first way heavier?
>>>
>>>
>>> About the meeting
>>>
>>> I thought maybe email discussion would be better?
>>> The discussion can be open for a few days and all interested people
>> in the community can participate, not just a few of us.
>>> And due to time zone and language issues, many people are hard to
>> attending the meetings, email discussions may be more suitable for these
>> people.
>>>
>>> What do you think?
>>>
>>> Best Regards,
>>> Zhiguo Wu
>>>
>>>> On Jun 20, 2022, at 23:15, Battula, Brahma Reddy
>> <bb...@visa.com.INVALID> wrote:
>>>>
>>>>>> Assuming the understanding above is correct, my new concerns are:
>>>>
>>>> Yes, Hope we'll in same page now. Still we can discuss in weekly
>> call.
>>>>
>>>>>> * Can we still call it as "HDP"? Doesn't it infringe Cloudera's
>> trademark?
>>>>>> If so, should we give a new stack name to it, as you mentioned as
>>>>>> planA in [3]?
>>>>>> (or it makes upgrading HDP cluster difficult? I don't understand
>> it so much)
>>>>
>>>> Yes, it may be. Hence I introduced the planA and preferred it. one
>> more idea(bad) came in my mind that can we treat "HDP: Hadoop Data
>> Platform", not sure whether it should be ok or not(and we might not
>> delivering as distro's).
>>>>
>>>>>> * It may be a bit tough work to fix Bigtop's build scripts so
>> that its
>>>>>> artifacts match Ambari's requirements.
>>>>>> (package versioning, file layouts, included and not included
>> files,
>>>>>> file permissions, user/group creation, etc.)
>>>>
>>>> Only paths matter right everything else can be re-used from the
>> hdp-select.
>>>>
>>>>
>>>> On 20/06/22, 8:13 PM, "Kengo Seki" <sekikn@apache.org <
>> ma...@apache.org>> wrote:
>>>>
>>>> Thanks for the elaboration Brahma, but I'm a bit confused.
>>>> At this point, my understanding is as follows. Are these correct?
>>>>
>>>> * In the 2.7.x line, we'll still use a stack definition based on
>> HDP's one [1].
>>>>
>>>> * But regarding RPM and DEB packages, we'll build them from source
>> by leveraging
>>>> Bigtop's build scripts and publish them on the ASF distribution
>> site
>>>> or somewhere.
>>>> repoinfo.xml [2] will also be fixed to point that repository.
>>>>
>>>> Assuming the understanding above is correct, my new concerns are:
>>>>
>>>> * Can we still call it as "HDP"? Doesn't it infringe Cloudera's
>> trademark?
>>>> If so, should we give a new stack name to it, as you mentioned as
>>>> planA in [3]?
>>>> (or it makes upgrading HDP cluster difficult? I don't understand
>> it so much)
>>>>
>>>> * It may be a bit tough work to fix Bigtop's build scripts so that
>> its
>>>> artifacts match Ambari's requirements.
>>>> (package versioning, file layouts, included and not included files,
>>>> file permissions, user/group creation, etc.)
>>>> I'm not sure if it's faster than developing Mpack.
>>>>
>>>> [1]:
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dFX07wfIY8aukfEUcJe9GN0rds8t3%2Fpenmsiszglkn4%3D&reserved=0
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dFX07wfIY8aukfEUcJe9GN0rds8t3%2Fpenmsiszglkn4%3D&reserved=0
>>>
>>>> [2]:
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP%2F2.6%2Frepos%2Frepoinfo.xml&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MDz57IloHfWYWcwmAI9h1pYDG45v1UbU6K2M%2BkpcDEA%3D&reserved=0
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP%2F2.6%2Frepos%2Frepoinfo.xml&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MDz57IloHfWYWcwmAI9h1pYDG45v1UbU6K2M%2BkpcDEA%3D&reserved=0
>>>
>>>> [3]:
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm77lhzo0njr2dhoock5twtm0c19j2py8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=P1pPHwM1YggdPT9qD4aAvJdehjjZdZ8mLhvn0ZfR7Gw%3D&reserved=0
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm77lhzo0njr2dhoock5twtm0c19j2py8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=P1pPHwM1YggdPT9qD4aAvJdehjjZdZ8mLhvn0ZfR7Gw%3D&reserved=0
>>>
>>>>
>>>> Kengo Seki <sekikn@gmail.com <ma...@gmail.com>>
>>>>
>>>> On Mon, Jun 20, 2022 at 8:05 PM Battula, Brahma Reddy
>>>> <bbattula@visa.com.invalid <ma...@visa.com.invalid>> wrote:
>>>>>
>>>>> We'll not use the HDP Packages (Only selects we'll use from hdp,
>> which can be plugged in bigtop). Still we'll use opensource packages and
>> bigtop to create the RPM's and selects.
>>>>> Just we can use the source tar balls opensource components(e.g
>> Hadoop-3.2.3 source tarballs from apache repo) and build rpm's and publish
>> in following repo. Or from the apache github repo's.
>>>>>
>>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2Fbigtop%2Fbigtop-3.1.0%2Frepos%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RyNyy7fOUlWBSFMJdd1iRMrmlKOVyA5NEGuMpSkCSEs%3D&reserved=0
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2Fbigtop%2Fbigtop-3.1.0%2Frepos%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RyNyy7fOUlWBSFMJdd1iRMrmlKOVyA5NEGuMpSkCSEs%3D&reserved=0
>>>
>>>>>
>>>>> Planning to have sync up call on this week, once it's finalize,
>> we can summarise here the approach. What do you think..?
>>>>>
>>>>>
>>>>> On 20/06/22, 4:06 PM, "Kengo Seki" <sekikn@apache.org <
>> ma...@apache.org>> wrote:
>>>>>
>>>>> Thanks for the reply, Brahma!
>>>>>
>>>>>> We'll use following repo, how other components in Apache are
>> hosted.
>>>>>
>>>>> Using that site for distribution has no problem. My concern is,
>> do we
>>>>> have the source tree for building HDP packages and does it still
>> work
>>>>> without the artifacts behind the paywall?
>>>>> According to ASF's release policy [1], we must publish the
>>>>> corresponding source releases whenever we publish binary
>> artifacts,
>>>>> such as RPM and DEB packages for HDP.
>>>>>
>>>>> [1]:
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Flegal%2Frelease-policy.html%23compiled-packages&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qPEbEEpIN8NCsrMnxW5xKbB%2B19bDJprq%2FiXjIF09vls%3D&reserved=0
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Flegal%2Frelease-policy.html%23compiled-packages&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qPEbEEpIN8NCsrMnxW5xKbB%2B19bDJprq%2FiXjIF09vls%3D&reserved=0
>>>
>>>>>
>>>>> Kengo Seki <sekikn@apache.org <ma...@apache.org>>
>>>>>
>>>>> On Mon, Jun 20, 2022 at 6:19 PM Battula, Brahma Reddy
>>>>> <bbattula@visa.com.invalid <ma...@visa.com.invalid>> wrote:
>>>>>>
>>>>>> Hi Kengo Seki,
>>>>>>
>>>>>> Thanks for pitching here.. We'll use following repo, how other
>> components in Apache are hosted. Any other suggestions..?
>>>>>>
>>>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OUGhE45KsDvPum4%2Fd9cwlJPfH66OlKE%2BgD557%2BiiAC8%3D&reserved=0
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OUGhE45KsDvPum4%2Fd9cwlJPfH66OlKE%2BgD557%2BiiAC8%3D&reserved=0
>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 20/06/22, 2:32 PM, "Kengo Seki" <sekikn@apache.org <
>> ma...@apache.org>> wrote:
>>>>>>
>>>>>> Oh, I've just noticed that I submitted an additional question
>> into the
>>>>>> wrong thread mistakenly.
>>>>>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fj6kc2h1lphvjvsblcthcl5yzzmjk3tb5&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UTDhFu1Bloa3FzD0shWWHoDt%2FrQD22JziV1%2BNAWp8NQ%3D&reserved=0
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fj6kc2h1lphvjvsblcthcl5yzzmjk3tb5&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UTDhFu1Bloa3FzD0shWWHoDt%2FrQD22JziV1%2BNAWp8NQ%3D&reserved=0>
>> is
>>>>>> originally intended as a reply to this thread. Sorry for the
>>>>>> confusion.
>>>>>>
>>>>>> Kengo Seki <sekikn@apache.org <ma...@apache.org>>
>>>>>>
>>>>>> On Mon, Jun 20, 2022 at 11:31 AM Kengo Seki <sekikn@apache.org <
>> ma...@apache.org>> wrote:
>>>>>>>
>>>>>>> Thank you so much for writing the draft and commenting on it,
>> Brahma and Viraj!
>>>>>>> Firstly, a minor comment:
>>>>>>>
>>>>>>>> Trunk Code : Parallel we can work for the trunk code, this we
>> might give as 2.8.
>>>>>>>> * Working on bigtop mpack
>>>>>>>
>>>>>>> This means adding a new built-in stack which uses the Bigtop
>>>>>>> repository, and not developing Mpack on the Bigtop side, right?
>>>>>>> If so, "bigtop mpack" sounds confusing a bit.
>>>>>>>
>>>>>>>> hdp-select can stay on branch-2.7 and it can be moved to
>> bigtop for trunk branch. Thoughts?
>>>>>>>
>>>>>>> Sounds good to me. During the 2.7.x releases, users can use the
>> HDP
>>>>>>> stack by default. In addition, they can also add the Bigtop
>> stack
>>>>>>> through the Mpack, as Zhiguo mentioned in [1].
>>>>>>> Since 2.8 or 3.0, we will drop HDP and users can use the
>> built-in
>>>>>>> Bigtop stack by default. It also allows us to drop the Mpack on
>> the
>>>>>>> Bigtop side.
>>>>>>>
>>>>>>> Regarding the built-in Bigtop stack developed on trunk, the
>> current
>>>>>>> BIGTOP stack in Ambari [2] may be a good start point to be
>> upgraded.
>>>>>>> I ensured that Ambari 2.7.6 was successfully built with the
>>>>>>> `-Dstack.distribution=BIGTOP` option, though Bigtop 0.8 is too
>> old to
>>>>>>> use for now.
>>>>>>>
>>>>>>> [1]:
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iqQ6QCM1%2BMYJ61tU%2FFk%2BWC8GvftYYGhrnu6V%2FFJ3d7o%3D&reserved=0
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iqQ6QCM1%2BMYJ61tU%2FFk%2BWC8GvftYYGhrnu6V%2FFJ3d7o%3D&reserved=0
>>>
>>>>>>> [2]:
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=FR5NTHCZt%2FQcqy9NyT63Q3NIYlEeJ6sIpbOqzfDOMRY%3D&reserved=0
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
>> [message truncated...]
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
>> For additional commands, e-mail: dev-help@ambari.apache.org
>>
>>
Re: Re: [Discuss] Initial Draft for Roadmap
Posted by Jun HE <ju...@apache.org>.
I agree with "1. Fix some security problems by upgrading Spring Framework
and others".Security is important and we can use this as a ramp-up as the
community is gradually bringing everything in Ambari back on track, like PR
review, CI jobs,...
For other things like "frontend refactor", it would be good to have more
inputs on existing issues/limitations before we decide to "refactor".
For the mpack part, my understanding is that "management packs" is a
framework that allows different stack vendors to provide a compatible stack
that can be maintained and operated by Ambari. There could be minor
differences between different vendors. So the definition like
path/repo/name etc should be the vendor's decision instead of being defined
by Ambari. I would suggest leaving the mpack related work to a certain
community, for example, Bigtop, and Ambari just review and accept their PR
just for the user's convenience.
Just my 2 cents. And happy to learn any comments.
吴治国 <wz...@163.com> 于2022年8月16日周二 17:50写道：
> Hi community,
>
> In my original idea, I was planning to provide Bigtop3.1.0 + Ambari2.7.5
> for users at first, and keep all big changes in Ambari2.8.0, e.g.
> 1、Fix some security problems by upgrading Spring Framework and others
> 2、Frontend refactor
> 3、Upgrade python version
> etc.
>
> All these changes will take long time, which may be bad for the release
> cycle of Ambari itself.
> So I'm considering we can move Mpack development to Ambari side, and 2.8.0
> is all about Bigtop stack, move other big changes to next major
> version(except the security problem, which absolutely need to be fixed in
> next release).
>
> But we need to make final decision for the things we discussed before,
> like,
> 1、Add prefix to installation path(/usr/bigtop)
> 2、Which repo to store the packages(with prefix in installation path)?
> 3、Related name(Stack name: BIGTOP, scripts: conf-select, distro-select,
> bigtop-select are all ok for me)?
> Any other things I forgot?
>
> What’s your opinion?
>
> Best Regards,
> Zhiguo Wu
>
> On 2022/06/23 06:26:44 "Battula, Brahma Reddy" wrote:
> >
> > IMO, First way is not heavier when we consider all the efforts for mpack
> with upgrades. And first one is easiest to adopt now.
> >
> > We'll start discussing the bigtop mailing list further.
> >
> >
> > On 21/06/22, 1:11 PM, "吴治国" <wz...@163.com> wrote:
> >
> > Thanks for the details, Brahma and Kengo.
> >
> > If I understand it correctly, there are two approaches under
> discussion:
> >
> > 1、Prepare to release 2.7.7, in this version, we won't use HDP
> components anymore, we'll replace it with the community version provided by
> Bigtop, and update RPM and DEB build scripts provided by Bigtop, change the
> installation path so hdp-select can be reused. (Due to component version
> changes, I don't know if the code under /stacks needs to be updated)
> > 2、Develop Bigtop mpack for Ambari 2.7.5, we don't need to change
> Ambari's code (if the code under /stacks can be reused in the first way,
> maybe it also works in this way?)
> >
> > Isn't the workload of the first way heavier?
> >
> >
> > About the meeting
> >
> > I thought maybe email discussion would be better?
> > The discussion can be open for a few days and all interested people
> in the community can participate, not just a few of us.
> > And due to time zone and language issues, many people are hard to
> attending the meetings, email discussions may be more suitable for these
> people.
> >
> > What do you think?
> >
> > Best Regards,
> > Zhiguo Wu
> >
> > > On Jun 20, 2022, at 23:15, Battula, Brahma Reddy
> <bb...@visa.com.INVALID> wrote:
> > >
> > >>> Assuming the understanding above is correct, my new concerns are:
> > >
> > > Yes, Hope we'll in same page now. Still we can discuss in weekly
> call.
> > >
> > >>> * Can we still call it as "HDP"? Doesn't it infringe Cloudera's
> trademark?
> > >>> If so, should we give a new stack name to it, as you mentioned as
> > >>> planA in [3]?
> > >>> (or it makes upgrading HDP cluster difficult? I don't understand
> it so much)
> > >
> > > Yes, it may be. Hence I introduced the planA and preferred it. one
> more idea(bad) came in my mind that can we treat "HDP: Hadoop Data
> Platform", not sure whether it should be ok or not(and we might not
> delivering as distro's).
> > >
> > >>> * It may be a bit tough work to fix Bigtop's build scripts so
> that its
> > >>> artifacts match Ambari's requirements.
> > >>> (package versioning, file layouts, included and not included
> files,
> > >>> file permissions, user/group creation, etc.)
> > >
> > > Only paths matter right everything else can be re-used from the
> hdp-select.
> > >
> > >
> > > On 20/06/22, 8:13 PM, "Kengo Seki" <sekikn@apache.org <
> ma...@apache.org>> wrote:
> > >
> > > Thanks for the elaboration Brahma, but I'm a bit confused.
> > > At this point, my understanding is as follows. Are these correct?
> > >
> > > * In the 2.7.x line, we'll still use a stack definition based on
> HDP's one [1].
> > >
> > > * But regarding RPM and DEB packages, we'll build them from source
> by leveraging
> > > Bigtop's build scripts and publish them on the ASF distribution
> site
> > > or somewhere.
> > > repoinfo.xml [2] will also be fixed to point that repository.
> > >
> > > Assuming the understanding above is correct, my new concerns are:
> > >
> > > * Can we still call it as "HDP"? Doesn't it infringe Cloudera's
> trademark?
> > > If so, should we give a new stack name to it, as you mentioned as
> > > planA in [3]?
> > > (or it makes upgrading HDP cluster difficult? I don't understand
> it so much)
> > >
> > > * It may be a bit tough work to fix Bigtop's build scripts so that
> its
> > > artifacts match Ambari's requirements.
> > > (package versioning, file layouts, included and not included files,
> > > file permissions, user/group creation, etc.)
> > > I'm not sure if it's faster than developing Mpack.
> > >
> > > [1]:
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dFX07wfIY8aukfEUcJe9GN0rds8t3%2Fpenmsiszglkn4%3D&reserved=0
> <
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dFX07wfIY8aukfEUcJe9GN0rds8t3%2Fpenmsiszglkn4%3D&reserved=0
> >
> > > [2]:
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP%2F2.6%2Frepos%2Frepoinfo.xml&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MDz57IloHfWYWcwmAI9h1pYDG45v1UbU6K2M%2BkpcDEA%3D&reserved=0
> <
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP%2F2.6%2Frepos%2Frepoinfo.xml&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MDz57IloHfWYWcwmAI9h1pYDG45v1UbU6K2M%2BkpcDEA%3D&reserved=0
> >
> > > [3]:
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm77lhzo0njr2dhoock5twtm0c19j2py8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=P1pPHwM1YggdPT9qD4aAvJdehjjZdZ8mLhvn0ZfR7Gw%3D&reserved=0
> <
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm77lhzo0njr2dhoock5twtm0c19j2py8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=P1pPHwM1YggdPT9qD4aAvJdehjjZdZ8mLhvn0ZfR7Gw%3D&reserved=0
> >
> > >
> > > Kengo Seki <sekikn@gmail.com <ma...@gmail.com>>
> > >
> > > On Mon, Jun 20, 2022 at 8:05 PM Battula, Brahma Reddy
> > > <bbattula@visa.com.invalid <ma...@visa.com.invalid>> wrote:
> > >>
> > >> We'll not use the HDP Packages (Only selects we'll use from hdp,
> which can be plugged in bigtop). Still we'll use opensource packages and
> bigtop to create the RPM's and selects.
> > >> Just we can use the source tar balls opensource components(e.g
> Hadoop-3.2.3 source tarballs from apache repo) and build rpm's and publish
> in following repo. Or from the apache github repo's.
> > >>
> > >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2Fbigtop%2Fbigtop-3.1.0%2Frepos%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RyNyy7fOUlWBSFMJdd1iRMrmlKOVyA5NEGuMpSkCSEs%3D&reserved=0
> <
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2Fbigtop%2Fbigtop-3.1.0%2Frepos%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RyNyy7fOUlWBSFMJdd1iRMrmlKOVyA5NEGuMpSkCSEs%3D&reserved=0
> >
> > >>
> > >> Planning to have sync up call on this week, once it's finalize,
> we can summarise here the approach. What do you think..?
> > >>
> > >>
> > >> On 20/06/22, 4:06 PM, "Kengo Seki" <sekikn@apache.org <
> ma...@apache.org>> wrote:
> > >>
> > >> Thanks for the reply, Brahma!
> > >>
> > >>> We'll use following repo, how other components in Apache are
> hosted.
> > >>
> > >> Using that site for distribution has no problem. My concern is,
> do we
> > >> have the source tree for building HDP packages and does it still
> work
> > >> without the artifacts behind the paywall?
> > >> According to ASF's release policy [1], we must publish the
> > >> corresponding source releases whenever we publish binary
> artifacts,
> > >> such as RPM and DEB packages for HDP.
> > >>
> > >> [1]:
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Flegal%2Frelease-policy.html%23compiled-packages&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qPEbEEpIN8NCsrMnxW5xKbB%2B19bDJprq%2FiXjIF09vls%3D&reserved=0
> <
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Flegal%2Frelease-policy.html%23compiled-packages&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qPEbEEpIN8NCsrMnxW5xKbB%2B19bDJprq%2FiXjIF09vls%3D&reserved=0
> >
> > >>
> > >> Kengo Seki <sekikn@apache.org <ma...@apache.org>>
> > >>
> > >> On Mon, Jun 20, 2022 at 6:19 PM Battula, Brahma Reddy
> > >> <bbattula@visa.com.invalid <ma...@visa.com.invalid>> wrote:
> > >>>
> > >>> Hi Kengo Seki,
> > >>>
> > >>> Thanks for pitching here.. We'll use following repo, how other
> components in Apache are hosted. Any other suggestions..?
> > >>>
> > >>>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OUGhE45KsDvPum4%2Fd9cwlJPfH66OlKE%2BgD557%2BiiAC8%3D&reserved=0
> <
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OUGhE45KsDvPum4%2Fd9cwlJPfH66OlKE%2BgD557%2BiiAC8%3D&reserved=0
> >
> > >>>
> > >>>
> > >>>
> > >>> On 20/06/22, 2:32 PM, "Kengo Seki" <sekikn@apache.org <
> ma...@apache.org>> wrote:
> > >>>
> > >>> Oh, I've just noticed that I submitted an additional question
> into the
> > >>> wrong thread mistakenly.
> > >>>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fj6kc2h1lphvjvsblcthcl5yzzmjk3tb5&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UTDhFu1Bloa3FzD0shWWHoDt%2FrQD22JziV1%2BNAWp8NQ%3D&reserved=0
> <
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fj6kc2h1lphvjvsblcthcl5yzzmjk3tb5&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UTDhFu1Bloa3FzD0shWWHoDt%2FrQD22JziV1%2BNAWp8NQ%3D&reserved=0>
> is
> > >>> originally intended as a reply to this thread. Sorry for the
> > >>> confusion.
> > >>>
> > >>> Kengo Seki <sekikn@apache.org <ma...@apache.org>>
> > >>>
> > >>> On Mon, Jun 20, 2022 at 11:31 AM Kengo Seki <sekikn@apache.org <
> ma...@apache.org>> wrote:
> > >>>>
> > >>>> Thank you so much for writing the draft and commenting on it,
> Brahma and Viraj!
> > >>>> Firstly, a minor comment:
> > >>>>
> > >>>>> Trunk Code : Parallel we can work for the trunk code, this we
> might give as 2.8.
> > >>>>> * Working on bigtop mpack
> > >>>>
> > >>>> This means adding a new built-in stack which uses the Bigtop
> > >>>> repository, and not developing Mpack on the Bigtop side, right?
> > >>>> If so, "bigtop mpack" sounds confusing a bit.
> > >>>>
> > >>>>> hdp-select can stay on branch-2.7 and it can be moved to
> bigtop for trunk branch. Thoughts?
> > >>>>
> > >>>> Sounds good to me. During the 2.7.x releases, users can use the
> HDP
> > >>>> stack by default. In addition, they can also add the Bigtop
> stack
> > >>>> through the Mpack, as Zhiguo mentioned in [1].
> > >>>> Since 2.8 or 3.0, we will drop HDP and users can use the
> built-in
> > >>>> Bigtop stack by default. It also allows us to drop the Mpack on
> the
> > >>>> Bigtop side.
> > >>>>
> > >>>> Regarding the built-in Bigtop stack developed on trunk, the
> current
> > >>>> BIGTOP stack in Ambari [2] may be a good start point to be
> upgraded.
> > >>>> I ensured that Ambari 2.7.6 was successfully built with the
> > >>>> `-Dstack.distribution=BIGTOP` option, though Bigtop 0.8 is too
> old to
> > >>>> use for now.
> > >>>>
> > >>>> [1]:
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iqQ6QCM1%2BMYJ61tU%2FFk%2BWC8GvftYYGhrnu6V%2FFJ3d7o%3D&reserved=0
> <
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iqQ6QCM1%2BMYJ61tU%2FFk%2BWC8GvftYYGhrnu6V%2FFJ3d7o%3D&reserved=0
> >
> > >>>> [2]:
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=FR5NTHCZt%2FQcqy9NyT63Q3NIYlEeJ6sIpbOqzfDOMRY%3D&reserved=0
> <
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
> [message truncated...]
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
> For additional commands, e-mail: dev-help@ambari.apache.org
>
>
Re: Re: [Discuss] Initial Draft for Roadmap
Posted by Jun HE <ju...@apache.org>.
I agree with "1. Fix some security problems by upgrading Spring Framework
and others".Security is important and we can use this as a ramp-up as the
community is gradually bringing everything in Ambari back on track, like PR
review, CI jobs,...
For other things like "frontend refactor", it would be good to have more
inputs on existing issues/limitations before we decide to "refactor".
For the mpack part, my understanding is that "management packs" is a
framework that allows different stack vendors to provide a compatible stack
that can be maintained and operated by Ambari. There could be minor
differences between different vendors. So the definition like
path/repo/name etc should be the vendor's decision instead of being defined
by Ambari. I would suggest leaving the mpack related work to a certain
community, for example, Bigtop, and Ambari just review and accept their PR
just for the user's convenience.
Just my 2 cents. And happy to learn any comments.
吴治国 <wz...@163.com> 于2022年8月16日周二 17:50写道：
> Hi community,
>
> In my original idea, I was planning to provide Bigtop3.1.0 + Ambari2.7.5
> for users at first, and keep all big changes in Ambari2.8.0, e.g.
> 1、Fix some security problems by upgrading Spring Framework and others
> 2、Frontend refactor
> 3、Upgrade python version
> etc.
>
> All these changes will take long time, which may be bad for the release
> cycle of Ambari itself.
> So I'm considering we can move Mpack development to Ambari side, and 2.8.0
> is all about Bigtop stack, move other big changes to next major
> version(except the security problem, which absolutely need to be fixed in
> next release).
>
> But we need to make final decision for the things we discussed before,
> like,
> 1、Add prefix to installation path(/usr/bigtop)
> 2、Which repo to store the packages(with prefix in installation path)?
> 3、Related name(Stack name: BIGTOP, scripts: conf-select, distro-select,
> bigtop-select are all ok for me)?
> Any other things I forgot?
>
> What’s your opinion?
>
> Best Regards,
> Zhiguo Wu
>
> On 2022/06/23 06:26:44 "Battula, Brahma Reddy" wrote:
> >
> > IMO, First way is not heavier when we consider all the efforts for mpack
> with upgrades. And first one is easiest to adopt now.
> >
> > We'll start discussing the bigtop mailing list further.
> >
> >
> > On 21/06/22, 1:11 PM, "吴治国" <wz...@163.com> wrote:
> >
> > Thanks for the details, Brahma and Kengo.
> >
> > If I understand it correctly, there are two approaches under
> discussion:
> >
> > 1、Prepare to release 2.7.7, in this version, we won't use HDP
> components anymore, we'll replace it with the community version provided by
> Bigtop, and update RPM and DEB build scripts provided by Bigtop, change the
> installation path so hdp-select can be reused. (Due to component version
> changes, I don't know if the code under /stacks needs to be updated)
> > 2、Develop Bigtop mpack for Ambari 2.7.5, we don't need to change
> Ambari's code (if the code under /stacks can be reused in the first way,
> maybe it also works in this way?)
> >
> > Isn't the workload of the first way heavier?
> >
> >
> > About the meeting
> >
> > I thought maybe email discussion would be better?
> > The discussion can be open for a few days and all interested people
> in the community can participate, not just a few of us.
> > And due to time zone and language issues, many people are hard to
> attending the meetings, email discussions may be more suitable for these
> people.
> >
> > What do you think?
> >
> > Best Regards,
> > Zhiguo Wu
> >
> > > On Jun 20, 2022, at 23:15, Battula, Brahma Reddy
> <bb...@visa.com.INVALID> wrote:
> > >
> > >>> Assuming the understanding above is correct, my new concerns are:
> > >
> > > Yes, Hope we'll in same page now. Still we can discuss in weekly
> call.
> > >
> > >>> * Can we still call it as "HDP"? Doesn't it infringe Cloudera's
> trademark?
> > >>> If so, should we give a new stack name to it, as you mentioned as
> > >>> planA in [3]?
> > >>> (or it makes upgrading HDP cluster difficult? I don't understand
> it so much)
> > >
> > > Yes, it may be. Hence I introduced the planA and preferred it. one
> more idea(bad) came in my mind that can we treat "HDP: Hadoop Data
> Platform", not sure whether it should be ok or not(and we might not
> delivering as distro's).
> > >
> > >>> * It may be a bit tough work to fix Bigtop's build scripts so
> that its
> > >>> artifacts match Ambari's requirements.
> > >>> (package versioning, file layouts, included and not included
> files,
> > >>> file permissions, user/group creation, etc.)
> > >
> > > Only paths matter right everything else can be re-used from the
> hdp-select.
> > >
> > >
> > > On 20/06/22, 8:13 PM, "Kengo Seki" <sekikn@apache.org <
> ma...@apache.org>> wrote:
> > >
> > > Thanks for the elaboration Brahma, but I'm a bit confused.
> > > At this point, my understanding is as follows. Are these correct?
> > >
> > > * In the 2.7.x line, we'll still use a stack definition based on
> HDP's one [1].
> > >
> > > * But regarding RPM and DEB packages, we'll build them from source
> by leveraging
> > > Bigtop's build scripts and publish them on the ASF distribution
> site
> > > or somewhere.
> > > repoinfo.xml [2] will also be fixed to point that repository.
> > >
> > > Assuming the understanding above is correct, my new concerns are:
> > >
> > > * Can we still call it as "HDP"? Doesn't it infringe Cloudera's
> trademark?
> > > If so, should we give a new stack name to it, as you mentioned as
> > > planA in [3]?
> > > (or it makes upgrading HDP cluster difficult? I don't understand
> it so much)
> > >
> > > * It may be a bit tough work to fix Bigtop's build scripts so that
> its
> > > artifacts match Ambari's requirements.
> > > (package versioning, file layouts, included and not included files,
> > > file permissions, user/group creation, etc.)
> > > I'm not sure if it's faster than developing Mpack.
> > >
> > > [1]:
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dFX07wfIY8aukfEUcJe9GN0rds8t3%2Fpenmsiszglkn4%3D&reserved=0
> <
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dFX07wfIY8aukfEUcJe9GN0rds8t3%2Fpenmsiszglkn4%3D&reserved=0
> >
> > > [2]:
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP%2F2.6%2Frepos%2Frepoinfo.xml&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MDz57IloHfWYWcwmAI9h1pYDG45v1UbU6K2M%2BkpcDEA%3D&reserved=0
> <
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Fblob%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FHDP%2F2.6%2Frepos%2Frepoinfo.xml&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MDz57IloHfWYWcwmAI9h1pYDG45v1UbU6K2M%2BkpcDEA%3D&reserved=0
> >
> > > [3]:
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm77lhzo0njr2dhoock5twtm0c19j2py8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=P1pPHwM1YggdPT9qD4aAvJdehjjZdZ8mLhvn0ZfR7Gw%3D&reserved=0
> <
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fm77lhzo0njr2dhoock5twtm0c19j2py8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=P1pPHwM1YggdPT9qD4aAvJdehjjZdZ8mLhvn0ZfR7Gw%3D&reserved=0
> >
> > >
> > > Kengo Seki <sekikn@gmail.com <ma...@gmail.com>>
> > >
> > > On Mon, Jun 20, 2022 at 8:05 PM Battula, Brahma Reddy
> > > <bbattula@visa.com.invalid <ma...@visa.com.invalid>> wrote:
> > >>
> > >> We'll not use the HDP Packages (Only selects we'll use from hdp,
> which can be plugged in bigtop). Still we'll use opensource packages and
> bigtop to create the RPM's and selects.
> > >> Just we can use the source tar balls opensource components(e.g
> Hadoop-3.2.3 source tarballs from apache repo) and build rpm's and publish
> in following repo. Or from the apache github repo's.
> > >>
> > >>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2Fbigtop%2Fbigtop-3.1.0%2Frepos%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RyNyy7fOUlWBSFMJdd1iRMrmlKOVyA5NEGuMpSkCSEs%3D&reserved=0
> <
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2Fbigtop%2Fbigtop-3.1.0%2Frepos%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RyNyy7fOUlWBSFMJdd1iRMrmlKOVyA5NEGuMpSkCSEs%3D&reserved=0
> >
> > >>
> > >> Planning to have sync up call on this week, once it's finalize,
> we can summarise here the approach. What do you think..?
> > >>
> > >>
> > >> On 20/06/22, 4:06 PM, "Kengo Seki" <sekikn@apache.org <
> ma...@apache.org>> wrote:
> > >>
> > >> Thanks for the reply, Brahma!
> > >>
> > >>> We'll use following repo, how other components in Apache are
> hosted.
> > >>
> > >> Using that site for distribution has no problem. My concern is,
> do we
> > >> have the source tree for building HDP packages and does it still
> work
> > >> without the artifacts behind the paywall?
> > >> According to ASF's release policy [1], we must publish the
> > >> corresponding source releases whenever we publish binary
> artifacts,
> > >> such as RPM and DEB packages for HDP.
> > >>
> > >> [1]:
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Flegal%2Frelease-policy.html%23compiled-packages&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qPEbEEpIN8NCsrMnxW5xKbB%2B19bDJprq%2FiXjIF09vls%3D&reserved=0
> <
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Flegal%2Frelease-policy.html%23compiled-packages&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qPEbEEpIN8NCsrMnxW5xKbB%2B19bDJprq%2FiXjIF09vls%3D&reserved=0
> >
> > >>
> > >> Kengo Seki <sekikn@apache.org <ma...@apache.org>>
> > >>
> > >> On Mon, Jun 20, 2022 at 6:19 PM Battula, Brahma Reddy
> > >> <bbattula@visa.com.invalid <ma...@visa.com.invalid>> wrote:
> > >>>
> > >>> Hi Kengo Seki,
> > >>>
> > >>> Thanks for pitching here.. We'll use following repo, how other
> components in Apache are hosted. Any other suggestions..?
> > >>>
> > >>>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OUGhE45KsDvPum4%2Fd9cwlJPfH66OlKE%2BgD557%2BiiAC8%3D&reserved=0
> <
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdlcdn.apache.org%2F&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OUGhE45KsDvPum4%2Fd9cwlJPfH66OlKE%2BgD557%2BiiAC8%3D&reserved=0
> >
> > >>>
> > >>>
> > >>>
> > >>> On 20/06/22, 2:32 PM, "Kengo Seki" <sekikn@apache.org <
> ma...@apache.org>> wrote:
> > >>>
> > >>> Oh, I've just noticed that I submitted an additional question
> into the
> > >>> wrong thread mistakenly.
> > >>>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fj6kc2h1lphvjvsblcthcl5yzzmjk3tb5&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UTDhFu1Bloa3FzD0shWWHoDt%2FrQD22JziV1%2BNAWp8NQ%3D&reserved=0
> <
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fj6kc2h1lphvjvsblcthcl5yzzmjk3tb5&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UTDhFu1Bloa3FzD0shWWHoDt%2FrQD22JziV1%2BNAWp8NQ%3D&reserved=0>
> is
> > >>> originally intended as a reply to this thread. Sorry for the
> > >>> confusion.
> > >>>
> > >>> Kengo Seki <sekikn@apache.org <ma...@apache.org>>
> > >>>
> > >>> On Mon, Jun 20, 2022 at 11:31 AM Kengo Seki <sekikn@apache.org <
> ma...@apache.org>> wrote:
> > >>>>
> > >>>> Thank you so much for writing the draft and commenting on it,
> Brahma and Viraj!
> > >>>> Firstly, a minor comment:
> > >>>>
> > >>>>> Trunk Code : Parallel we can work for the trunk code, this we
> might give as 2.8.
> > >>>>> * Working on bigtop mpack
> > >>>>
> > >>>> This means adding a new built-in stack which uses the Bigtop
> > >>>> repository, and not developing Mpack on the Bigtop side, right?
> > >>>> If so, "bigtop mpack" sounds confusing a bit.
> > >>>>
> > >>>>> hdp-select can stay on branch-2.7 and it can be moved to
> bigtop for trunk branch. Thoughts?
> > >>>>
> > >>>> Sounds good to me. During the 2.7.x releases, users can use the
> HDP
> > >>>> stack by default. In addition, they can also add the Bigtop
> stack
> > >>>> through the Mpack, as Zhiguo mentioned in [1].
> > >>>> Since 2.8 or 3.0, we will drop HDP and users can use the
> built-in
> > >>>> Bigtop stack by default. It also allows us to drop the Mpack on
> the
> > >>>> Bigtop side.
> > >>>>
> > >>>> Regarding the built-in Bigtop stack developed on trunk, the
> current
> > >>>> BIGTOP stack in Ambari [2] may be a good start point to be
> upgraded.
> > >>>> I ensured that Ambari 2.7.6 was successfully built with the
> > >>>> `-Dstack.distribution=BIGTOP` option, though Bigtop 0.8 is too
> old to
> > >>>> use for now.
> > >>>>
> > >>>> [1]:
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iqQ6QCM1%2BMYJ61tU%2FFk%2BWC8GvftYYGhrnu6V%2FFJ3d7o%3D&reserved=0
> <
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iqQ6QCM1%2BMYJ61tU%2FFk%2BWC8GvftYYGhrnu6V%2FFJ3d7o%3D&reserved=0
> >
> > >>>> [2]:
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=FR5NTHCZt%2FQcqy9NyT63Q3NIYlEeJ6sIpbOqzfDOMRY%3D&reserved=0
> <
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7Cf26dc731668c471ef20008da53597102%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913940886207699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
> [message truncated...]
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
> For additional commands, e-mail: dev-help@ambari.apache.org
>
>