You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by GitBox <gi...@apache.org> on 2021/02/08 22:13:35 UTC
[GitHub] [ambari] sziszo opened a new pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
sziszo opened a new pull request #3283:
URL: https://github.com/apache/ambari/pull/3283
Change-Id: I7a560a7c2d6ff6c3681fc384d51c23bbb36330c8
## What changes were proposed in this pull request?
Public HDP binaries will no longer available: https://my.cloudera.com/knowledge/Cloudera-Customer-Advisory-Paywall-Update-External?id=306085
Please analyse and eliminate(replace) the necessary references from the codebase where rpms and tarballs of HDP are accessible.
Additionally non managed HDP tarballs should be replaced with open source ones. For example: https://github.com/apache/ambari/blob/branch-2.7/ambari-metrics/pom.xml#L43
## How was this patch tested?
the patch was tested manually
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] NISHANTSHRIVASTAV commented on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
NISHANTSHRIVASTAV commented on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-776715968
> > Yes, How can we build source code of apache ambari if we need public HDP repositories which are behind the paywall now?
>
> This PR replaces the used HDP components with their Apache counterparts, so can build Ambari from source
@sziszo Great..! I am able to build rpm for apache ambari server and ambari agent. I started ambari server but there is no **Version Definition File** in Select Version section in apache ambari user interface.
How can we create version definition file with working public repositories for installing custom stack?
**Please let me know from where we can get install HDP like stack .**
Thank you for your time & consideration..!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] sburn edited a comment on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
sburn edited a comment on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-776549642
Have you tested it? This probably won't work because RPMs in Hortons' repos were built with the specific structure on the filesystem and other dependencies of the HDP stack Ambari works with.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] NISHANTSHRIVASTAV commented on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
NISHANTSHRIVASTAV commented on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-776484527
Yes, How can we build source code of apache ambari if we need public HDP repositories which are behind the paywall now?
For example:
https://github.com/apache/ambari/blob/branch-2.7/ambari-metrics/pom.xml#L43
https://github.com/apache/ambari/blob/branch-2.7/ambari-metrics/pom.xml#L45
https://github.com/apache/ambari/blob/branch-2.7/ambari-metrics/pom.xml#L49
So, Are there not any public repositories instead of HDP repositories? and if we build apache ambari from source do we get all services like HDFS, Spark, NiFi etc complete stack in apache ambari or we have to build it on our own stack of components?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] hapylestat commented on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
hapylestat commented on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-784561579
At Ambari build stage(especially Ambari Metrics) we don't use RPM's for hadoop services, but tarballs which come with unified structure and not depend on who build them.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] jguilhermemv commented on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
jguilhermemv commented on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-819835000
This PR was merged into branch-2.7, but the release version doesn't contain this merge. Also, the gzip file for distribution found in https://cwiki.apache.org/confluence/display/AMBARI/Installation+Guide+for+Ambari+2.7.5 it's not update also. That said, one has to download this specific branch to install and configure Ambari.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] sziszo edited a comment on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
sziszo edited a comment on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-776511988
> Yes, How can we build source code of apache ambari if we need public HDP repositories which are behind the paywall now?
This PR replaces the used HDP components with their Apache counterparts, so can build Ambari from source
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] boy-xiaozhang commented on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
boy-xiaozhang commented on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-864371768
> > > > > Yes, How can we build source code of apache ambari if we need public HDP repositories which are behind the paywall now?
> > > >
> > > >
> > > > This PR replaces the used HDP components with their Apache counterparts, so can build Ambari from source
> > >
> > >
> > > @sziszo Great..! I am able to build rpm for apache ambari server and ambari agent. I started ambari server but there is no **Version Definition File** in Select Version section in apache ambari user interface.
> > > How can we create version definition file with working public repositories for installing custom stack?
> > > **Please let me know from where we can get install HDP like stack .**
> > > Thank you for your time & consideration..!
> >
> >
> > @NISHANTSHRIVASTAV , are you find someone free HDP repo to use with Ambari?
>
> No I have not got open source HDP repositores.
At present, I have also encountered this problem. Do you have a good solution
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] sziszo commented on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
sziszo commented on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-776508281
> @sziszo , are you able to build RPMs with the new binaries ?
@payert yes, I can build the RPMs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] sziszo commented on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
sziszo commented on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-775499897
@payert, @hapylestat could you please review this PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] sburn edited a comment on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
sburn edited a comment on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-776549642
Have you tested it? This probably won't work because apps in Hortons' repos are built with the specific structure on the filesystem and other dependencies of the HDP stack Ambari works with.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] hurryupharry commented on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
hurryupharry commented on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-903875346
@jguilhermemv it seems an effort is already underway. If you join the Ambari user (not dev) email list, there have recently been conversations about this. If you join the list then send an email to user-thread.4441@ambari.apache.org you will receive a reply containing all replies to the thread. That is the main thread that I saw that discusses the present situation, and people's plans/efforts to move past it. There is much talk of collaboration etc. Hope this helps.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] NISHANTSHRIVASTAV commented on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
NISHANTSHRIVASTAV commented on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-855818492
> > > > Yes, How can we build source code of apache ambari if we need public HDP repositories which are behind the paywall now?
> > >
> > >
> > > This PR replaces the used HDP components with their Apache counterparts, so can build Ambari from source
> >
> > @sziszo Great..! I am able to build rpm for apache ambari server and ambari agent. I started ambari server but there is no **Version Definition File** in Select Version section in apache ambari user interface.
> >
> > How can we create version definition file with working public repositories for installing custom stack?
> > **Please let me know from where we can get install HDP like stack .**
> >
> > Thank you for your time & consideration..!
>
> @NISHANTSHRIVASTAV , are you find someone free HDP repo to use with Ambari?
>
>
No I have not got open source HDP repositores.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] sburn commented on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
sburn commented on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-776549642
Have you tested it? This probably won't work because apps in Hortons' repos are built with the specific structure on the filesystem and other dependencies of the HDP stack.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] hurryupharry commented on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
hurryupharry commented on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-897655121
> > > > > > Yes, How can we build source code of apache ambari if we need public HDP repositories which are behind the paywall now?
> > > > >
> > > > >
> > > > > This PR replaces the used HDP components with their Apache counterparts, so can build Ambari from source
> > > >
> > > >
> > > > @sziszo Great..! I am able to build rpm for apache ambari server and ambari agent. I started ambari server but there is no **Version Definition File** in Select Version section in apache ambari user interface.
> > > > How can we create version definition file with working public repositories for installing custom stack?
> > > > **Please let me know from where we can get install HDP like stack .**
> > > > Thank you for your time & consideration..!
> > >
> > >
> > > @NISHANTSHRIVASTAV , are you find someone free HDP repo to use with Ambari?
> >
> >
> > No I have not got open source HDP repositores.
>
> At present, I have also encountered this problem. Do you have a good solution
Same here - I've spent hours over the past few days getting Ambari to compile and install on Ubuntu 18.04. This morning I finally had it working, but I immediately ran into this problem with no available versions in the wizard.
This PR acknowledges the problem that HDP repos are no longer an option since Hortonworks does not exist anymore, but only fixes the Ambari build problems.
@NISHANTSHRIVASTAV has repeatedly and clearly asked the question here: how do we get repositories for Hadoop, Hbase, Solr, Kafka etc so that they can be made available in the Ambari Cluster creation wizard?
I don't mind if it involves work on my part, but it is beyond my current knowledge, I would need some documentation on how to achieve this.
Currently it seems that Ambari is not a viable product? I'm really interested in getting a fully open source cluster operational but that would have to include a management piece like Ambari.
So, to repeat @NISHANTSHRIVASTAV's question, has anyone out there built an operational Ambari recently with repositories for HBase, Hadoop, Kafka etc that can be made to appear in the 'select version' section of the Wizard, and is there documentation on how to do this? If not, of what use now is Ambari? Not trying to be confrontational, I really want to use Ambari, but it seems not possible without having way more knowledge than I currently possess on setting up repositories etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] jguilhermemv commented on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
jguilhermemv commented on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-903315382
@hurryupharry what do you think of me, you and @NISHANTSHRIVASTAV work together to build all the parts of Ambari together and host it somewhere so we can have binaries to install and configure a full open source hadoop cluster?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@ambari.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] payert commented on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
payert commented on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-775956894
@sziszo , are you able to build RPMs with the new binaries ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] alexsandrohaag commented on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
alexsandrohaag commented on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-849568817
> > > Yes, How can we build source code of apache ambari if we need public HDP repositories which are behind the paywall now?
> >
> >
> > This PR replaces the used HDP components with their Apache counterparts, so can build Ambari from source
>
> @sziszo Great..! I am able to build rpm for apache ambari server and ambari agent. I started ambari server but there is no **Version Definition File** in Select Version section in apache ambari user interface.
>
> How can we create version definition file with working public repositories for installing custom stack?
> **Please let me know from where we can get install HDP like stack .**
>
> Thank you for your time & consideration..!
@NISHANTSHRIVASTAV , are you find someone free HDP repo to use with Ambari?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] payert merged pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
payert merged pull request #3283:
URL: https://github.com/apache/ambari/pull/3283
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] sziszo commented on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
sziszo commented on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-776511988
> Yes, How can we build source code of apache ambari if we need public HDP repositories which are behind the paywall now?
this PR replaces the used HDP components with their Apache counterparts, so can build Ambari from source
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] getChan commented on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
getChan commented on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-779071307
It builds well, rpm install well, ambari-server start well.
BUT it can't proceed ambari installer step 1 (select version)
```log
// ambari-server.log
2021-02-15 17:54:12,366 INFO [ambari-client-thread-163] AmbariMetaInfo:1430 - Stack HDP-2.0.6 is not active, skipping VDF
2021-02-15 17:54:12,367 INFO [ambari-client-thread-163] AmbariMetaInfo:1430 - Stack HDP-2.0.6.GlusterFS is not active, skipping VDF
2021-02-15 17:54:12,367 INFO [ambari-client-thread-163] AmbariMetaInfo:1430 - Stack HDP-2.1 is not active, skipping VDF
2021-02-15 17:54:12,367 INFO [ambari-client-thread-163] AmbariMetaInfo:1430 - Stack HDP-2.1.GlusterFS is not active, skipping VDF
2021-02-15 17:54:12,367 INFO [ambari-client-thread-163] AmbariMetaInfo:1430 - Stack HDP-2.2 is not active, skipping VDF
2021-02-15 17:54:12,367 INFO [ambari-client-thread-163] AmbariMetaInfo:1430 - Stack HDP-2.3 is not active, skipping VDF
2021-02-15 17:54:12,368 INFO [ambari-client-thread-163] AmbariMetaInfo:1430 - Stack HDP-2.3.ECS is not active, skipping VDF
2021-02-15 17:54:12,368 INFO [ambari-client-thread-163] AmbariMetaInfo:1428 - Stack HDP-2.3.GlusterFS is not valid, skipping VDF: The service 'OOZIE' in stack 'HDP:2.3.GlusterFS' extends a non-existent service: 'common-services/OOZIE/5.0.0.2.3'
2021-02-15 17:54:12,368 INFO [ambari-client-thread-163] AmbariMetaInfo:1430 - Stack HDP-2.4 is not active, skipping VDF
2021-02-15 17:54:12,368 INFO [ambari-client-thread-163] AmbariMetaInfo:1430 - Stack HDP-2.5 is not active, skipping VDF
2021-02-15 17:54:12,368 INFO [ambari-client-thread-163] AmbariMetaInfo:1430 - Stack HDP-2.6 is not active, skipping VDF
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] sziszo commented on pull request #3283: AMBARI-25599 Consider to eliminate HDP public binary references (santal)
Posted by GitBox <gi...@apache.org>.
sziszo commented on pull request #3283:
URL: https://github.com/apache/ambari/pull/3283#issuecomment-775512365
retest
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org