You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ambari.apache.org by Matt Andruff <ma...@gmail.com> on 2020/05/11 14:14:59 UTC
life after Hortonworks
Hey I was just poking around today trying to figure out if ambari still
works to install a cluster.
I recently was able to install a Cloudera Manager installation.
I was about to start an Ambari installation but when I poke the repos they
all seem to need a user/password.
Is there a non-hortonworks(cloudera) hadoop that can be still be installed
with Amabari?
If there is documentation somewhere just let me know, I couldn't find the
answer googling as Hortonworks(cloudera) docs are all that come up.
--
Live life
Laugh Often
RE: [EXTERNAL] life after Hortonworks
Posted by "Woodcock, Michael W., M.B.A." <wo...@mayo.edu>.
All the new\old stuff is behind a Cloudera pay wall. I am not sure how much of it. Maybe someone has a mirror…
From: Matt Andruff [mailto:matt.andruff@gmail.com]
Sent: Monday, May 11, 2020 9:15 AM
To: user@ambari.apache.org
Subject: [EXTERNAL] life after Hortonworks
Hey I was just poking around today trying to figure out if ambari still works to install a cluster.
I recently was able to install a Cloudera Manager installation.
I was about to start an Ambari installation but when I poke the repos they all seem to need a user/password.
Is there a non-hortonworks(cloudera) hadoop that can be still be installed with Amabari?
If there is documentation somewhere just let me know, I couldn't find the answer googling as Hortonworks(cloudera) docs are all that come up.
--
Live life
Laugh Often
Re: life after Hortonworks
Posted by Matt Andruff <ma...@gmail.com>.
You know what, I think the effort being completed at Apache BigTop
is giving Ambari a fighting chance. I'm going to write something up on
Ambari and Bigtop of how to use an mpack to install Apache BigTop on
Ambari. (Basically a rip and replace for HDP)
I'll post back here in a week or so once it's up and running right now
there are pieces everywhere. If you want to follow progress read about it
here: BIGTOP-3264 <https://issues.apache.org/jira/browse/BIGTOP-3264>,
AMBARI-25366 <https://issues.apache.org/jira/browse/AMBARI-25366>. I'll
update my efforts to document the process in both Ambari getting started
and BigTop getting started. We aren't at a polished finished stage but
their has been a huge effort to get this off the ground. (And we can
likely steal a bunch of committed code from HDP to get some of this
moving faster)
Big thanks to @Ganesh Raju <ga...@linaro.org> for pointing me in
the right direction.
Actually kind of cool, as this would enable you to move at the pace of
Bigtop to a new version of [apache component] instead of always waiting for
a new release from cloudera. (FYI, Cloudera and Horton works both have
used BigTop to create releases in the past, just in case you don't know
BigTop)
And for those that wanted to have an open source option, BigTop is still
the only 100% open source option for RPMs. (and as far as I know isn't a
commercial company)
On Mon, May 11, 2020 at 1:08 PM Stephen Boesch <ja...@gmail.com> wrote:
> I am reading between the lines that ambari is no longer a strategic
> platform. Would someone please provide a link/reference to a Cloudera press
> release or blog describing this and maybe related decisions/roadmaps? thx!
>
> Am Mo., 11. Mai 2020 um 10:05 Uhr schrieb Aaron Bossert <
> aaron@punchcyber.com>:
>
>> For what it is worth, I have written blueprints before, but never
>> stacks. The documentation and tutorials for ambari stacks and blueprints
>> are horribly out of date, incomplete, or flat out missing. Perhaps that
>> could be an initial task for the community to undertake so that those of us
>> who were using the Hortonworks suite of tools and were comfortable with
>> Ambari can sever the cord, as it were....relying on commercial companies to
>> support open source tools once their objectives have changed is rarely a
>> good thing.
>>
>> Get Outlook for iOS <https://aka.ms/o0ukef>
>> ------------------------------
>> *From:* Ganesh Raju <ga...@linaro.org>
>> *Sent:* Monday, May 11, 2020 12:57:50 PM
>> *To:* user@ambari.apache.org <us...@ambari.apache.org>
>> *Subject:* Re: life after Hortonworks
>>
>> Here is more Apache Bigtop info
>>
>> releases
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FReleases&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=QqZETOUT42xKoDSVwdOFSgIS9cyS%2BeEBH12hM7e5gdI%3D&reserved=0>
>> mailinglist
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbigtop.apache.org%2Fmail-lists.html&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=EyqeVHJozqXHtFh76b7KQtJ4Tl%2FKhsHaVN6EBiW3ev0%3D&reserved=0>
>> list of components
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FBigtop%2B1.4.0%2BRelease&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=WmuQu3t9PBrITRGZnFCAm7i8dB0sxuqF%2BWHkOql75xc%3D&reserved=0>
>>
>> Thanks
>> Ganesh Raju
>>
>>
>> On Mon, May 11, 2020 at 10:00 AM Ganesh Raju <ga...@linaro.org>
>> wrote:
>>
>> Yes that's what Bigtop did. BTW, Apache Bigtop already has hadoop and 30
>> other components.
>>
>> Ganesh
>>
>> On Mon, May 11, 2020 at 9:56 AM Matt Andruff <ma...@gmail.com>
>> wrote:
>>
>> I kept digging and I found that ambari supports stacks (that we have all
>> seen before). You can write your own custom stack
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAMBARI%2FOverview%23Overview-Structure&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=BKlsv87RZhCVj83Y7WAUw0Dydo9jiqRIsYRpN0owXZk%3D&reserved=0>
>> meaning you could write the "glue" what apache projects hortonworks had
>> written to roll out a set of services. I guess this is how they got around
>> being open source. I guess I'll start looking into making my own stack
>> from scratch. If anyone already has one or know of one, please let me know.
>>
>> On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <ga...@linaro.org>
>> wrote:
>>
>> Apache Bigtop now has mpack and works with Ambari. You could try that.
>>
>> Thanks,
>> Ganesh
>>
>> On Mon, May 11, 2020 at 9:15 AM Matt Andruff <ma...@gmail.com>
>> wrote:
>>
>> Hey I was just poking around today trying to figure out if ambari still
>> works to install a cluster.
>>
>> I recently was able to install a Cloudera Manager installation.
>>
>> I was about to start an Ambari installation but when I poke the repos
>> they all seem to need a user/password.
>>
>> Is there a non-hortonworks(cloudera) hadoop that can be still be
>> installed with Amabari?
>>
>> If there is documentation somewhere just let me know, I couldn't find the
>> answer googling as Hortonworks(cloudera) docs are all that come up.
>>
>> --
>> Live life
>> Laugh Often
>>
>>
>>
>> --
>> IRC: ganeshraju@#linaro on irc.freenode.ne
>> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=78S3EpIVAqTe6BbaRCxjYMtghzttYoie%2FLHwYbFYCbc%3D&reserved=0>
>> t
>>
>>
>>
>> --
>> Live life
>> Laugh Often
>>
>>
>>
>> --
>> IRC: ganeshraju@#linaro on irc.freenode.ne
>> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=78S3EpIVAqTe6BbaRCxjYMtghzttYoie%2FLHwYbFYCbc%3D&reserved=0>
>> t
>>
>>
>>
>> --
>> IRC: ganeshraju@#linaro on irc.freenode.ne
>> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=78S3EpIVAqTe6BbaRCxjYMtghzttYoie%2FLHwYbFYCbc%3D&reserved=0>
>> t
>>
>
--
Live life
Laugh Often
Re: life after Hortonworks
Posted by Matt Andruff <ma...@gmail.com>.
@jeremy montgomery <al...@gmail.com> thanks that would be amazing
if you could add what you know. The docs are a little scarce.. I'll add
what I learn as well as I flush out BigTop mpack so Ambari can still have a
future.
On Thu, May 14, 2020 at 9:22 AM jeremy montgomery <al...@gmail.com>
wrote:
> @aaron - i don't think that the server/agent piece needs to be messed
> with. It is more puppet than ansible, but it works pretty well. It has a
> few issues that need to be worked out, but nothing insurmountable.
>
> Agent/Server (pieces that need work)
>
> - Ability to invalidate the agent cache.
> - Multithreading - Currently alerts/restarts work on a concept of
> threads and timeouts. You can end up with too many concurrent tasks/alerts
> and things get backlogged waiting on the timeouts. However, if you up the
> threads you run into a lot of race conditions with auto-restart and alerts
> firing concurrently. ----> main piece here is it needs a prioritized
> queuing system locally.
> - Order of restarts and hierarchy - Right now this is hard coded at
> the server level within a component and somewhat for full restarts based on
> the order on the left hand bar.
> - Need the ability to modify the full restart sequence
> - Need the ability to state whether a restart is blocking, blocking
> only when component is on the same server, and non-blocking.
> - Need the ability to chain 2 different components together.
> Example, if i restart a kerberized hiveserver2 i can have it restart hue as
> well after it is restarted. Otherwise hue will error out since the spnego
> hashes are different.
>
> I'll make some edits to the confluence page explaining how the
> stacks/mpacks/extensions work.
>
> On Tue, May 12, 2020 at 10:51 AM Matt Andruff <ma...@gmail.com>
> wrote:
>
>> @Aaron Bossert - For that type of change (Ansible) you probably need to
>> write up a design paper. (I have seen this done, and I'm not sure it's
>> required but seems to be common practice) Sounds like ansible would be a
>> good fit.
>>
>> @Jeremy - thanks for the insights, I might ping you if I get stuck trying
>> to get into it. I certainly wish there was a clean standard for
>> installation locations.
>>
>>
>>
>> On Tue, May 12, 2020 at 10:36 AM Aaron Bossert <aa...@punchcyber.com>
>> wrote:
>>
>>> Jeremy,
>>>
>>>
>>>
>>> I would be happy to tackle this with you and whoever else is
>>> willing…that being said, I have always had a love/hate relationship with
>>> Ambari…the stacks have always been intimidating to me….you mention
>>> revisiting the architecture….do you have any ideas for what might be
>>> better? In my opinion, I have always wanted to replace the guts of ambari
>>> with a system that is underpinned by Ansible…seems much more user-friendly
>>> and MUCH easier to configure and extend for those who are not comfortable
>>> with doing their own Java to Python and creating RPM’s and the like….What
>>> are your thoughts?
>>>
>>>
>>>
>>>
>>>
>>> -- M. Aaron Bossert
>>>
>>>
>>>
>>> *From: *jeremy montgomery <al...@gmail.com>
>>> *Reply-To: *"user@ambari.apache.org" <us...@ambari.apache.org>
>>> *Date: *Tuesday, May 12, 2020 at 9:14 AM
>>> *To: *"user@ambari.apache.org" <us...@ambari.apache.org>
>>> *Subject: *Re: life after Hortonworks
>>>
>>>
>>>
>>> I make mpacks all the time, but tend to push them as extensions because
>>> the stack architecture is byzantine.
>>>
>>> - It is java that deploys python code. As such, the structure
>>> behaves like java, which makes it a pain in the rear to track down simple
>>> python functions. Sometimes the code ends up in common-services, sometimes
>>> it stays in the current version of stacks. Some stack components only use
>>> code in their latest version, some have code files strung through the
>>> folders of 8 previous versions and 3 stacks (looking at you LLAP).
>>> - The install code doesn't know where it wants to be. sometimes its
>>> in the stack, sometimes it is in ambari. This means that a version of
>>> ambari tends to be hard coded to 1-2 versions of a stack.
>>> - Changing javascript for a stack isn't possible. This means that
>>> stack components are hard coded into the ember.js with a bunch of if
>>> statements.
>>> - Making additions to a stack feature is a major process (like
>>> adding hbase thrift to an existing hbase installation)
>>> - Upgrades require a yum file regardless of installation method. so
>>> learning how to create dummy rpms is necessary.
>>>
>>> I'd be willing to chip in and maintain the stack python code but the
>>> stack architecture really needs to be revisited.
>>>
>>>
>>>
>>> On Mon, May 11, 2020 at 1:50 PM Aaron Bossert <aa...@punchcyber.com>
>>> wrote:
>>>
>>> Matt,
>>>
>>>
>>>
>>> Yeah, my thought was to start with whatever the most recent HDP/HDF
>>> stack definition as a starting point. It just so happens that I have a
>>> backburner project to do this already. I have been using Hortonworks for a
>>> long time, but have found recently that I needed to install newer versions
>>> of Apache Druid and Apache Storm, which would require me to do a new
>>> stack…Full disclosure: I have NEVER mucked with stacks and am not a Python
>>> guy…I write in Scala/Java predominantly….that being said, I would be happy
>>> to collaborate on this if anyone feels that this would be worthwhile and
>>> useful to the broader community.
>>>
>>>
>>>
>>> -- M. Aaron Bossert
>>>
>>>
>>>
>>> *From: *Matt Andruff <ma...@gmail.com>
>>> *Reply-To: *"user@ambari.apache.org" <us...@ambari.apache.org>
>>> *Date: *Monday, May 11, 2020 at 1:46 PM
>>> *To: *"user@ambari.apache.org" <us...@ambari.apache.org>
>>> *Subject: *Re: life after Hortonworks
>>>
>>>
>>>
>>> Cloudera no longer uses ambari. They stuck with Cloudera manager in
>>> their release of CDP.(CDH+HDP=CDP)
>>>
>>>
>>>
>>> https://docs.cloudera.com/cdpdc/7.0/overview/topics/cdpdc-overview.html
>>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.cloudera.com%2Fcdpdc%2F7.0%2Foverview%2Ftopics%2Fcdpdc-overview.html&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=7%2BMaRSRcCIjjKVT777pp18IYYsXEAGk3VPdNuOexV2w%3D&reserved=0>
>>>
>>>
>>>
>>> I don't think this means that Ambari is dead. I do think it means as
>>> stated that the community will need to take on packaging a stack and
>>> building rpms.(or at least packaging the stack)
>>>
>>>
>>>
>>> The legacy code for stack of HDP 2.6 stream is out there already in the
>>> repo so it's just some work to create rpms of whatever Ambari wants to
>>> release.
>>>
>>>
>>>
>>> I assume the stacks section is so poorly documented is because
>>> Hortonworks was doing the work of packaging. I'm not sure the level of
>>> effort needed to make a stack work but it seems like we could start with
>>> the last HDP build (3.1.4) and keep moving forward.
>>>
>>>
>>>
>>>
>>>
>>> On Mon., May 11, 2020, 13:08 Stephen Boesch, <ja...@gmail.com> wrote:
>>>
>>> I am reading between the lines that ambari is no longer a strategic
>>> platform. Would someone please provide a link/reference to a Cloudera press
>>> release or blog describing this and maybe related decisions/roadmaps? thx!
>>>
>>>
>>>
>>> Am Mo., 11. Mai 2020 um 10:05 Uhr schrieb Aaron Bossert <
>>> aaron@punchcyber.com>:
>>>
>>> For what it is worth, I have written blueprints before, but never
>>> stacks. The documentation and tutorials for ambari stacks and blueprints
>>> are horribly out of date, incomplete, or flat out missing. Perhaps that
>>> could be an initial task for the community to undertake so that those of us
>>> who were using the Hortonworks suite of tools and were comfortable with
>>> Ambari can sever the cord, as it were....relying on commercial companies to
>>> support open source tools once their objectives have changed is rarely a
>>> good thing.
>>>
>>>
>>>
>>> Get Outlook for iOS
>>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=AqkexLKW6pLpkhG0auiGnGpIy5aZpcc%2B%2BG53O2qCNek%3D&reserved=0>
>>> ------------------------------
>>>
>>> *From:* Ganesh Raju <ga...@linaro.org>
>>> *Sent:* Monday, May 11, 2020 12:57:50 PM
>>> *To:* user@ambari.apache.org <us...@ambari.apache.org>
>>> *Subject:* Re: life after Hortonworks
>>>
>>>
>>>
>>> Here is more Apache Bigtop info
>>>
>>>
>>>
>>> releases
>>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FReleases&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=8xGs5j8Akzib7cRVlEgWq8ORcz8dWduDXqu8ev69mTE%3D&reserved=0>
>>>
>>> mailinglist
>>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbigtop.apache.org%2Fmail-lists.html&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=NgLr8gAPF2BXSr%2FXnUuaR2%2FdDU0c40RlyCoxiATR%2FWw%3D&reserved=0>
>>>
>>> list of components
>>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FBigtop%2B1.4.0%2BRelease&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=vXMOynon6RZFabIVAo5PQ4D%2B6cB9rq%2BHSBbYBnH9uPI%3D&reserved=0>
>>>
>>>
>>>
>>> Thanks
>>>
>>> Ganesh Raju
>>>
>>>
>>>
>>>
>>>
>>> On Mon, May 11, 2020 at 10:00 AM Ganesh Raju <ga...@linaro.org>
>>> wrote:
>>>
>>> Yes that's what Bigtop did. BTW, Apache Bigtop already has hadoop and 30
>>> other components.
>>>
>>>
>>>
>>> Ganesh
>>>
>>>
>>>
>>> On Mon, May 11, 2020 at 9:56 AM Matt Andruff <ma...@gmail.com>
>>> wrote:
>>>
>>> I kept digging and I found that ambari supports stacks (that we have all
>>> seen before). You can write your own custom stack
>>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAMBARI%2FOverview%23Overview-Structure&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=3itBtwJy1mH3Eu2FbsVx0WurYZFlTlVY7VtV5D%2FbDjg%3D&reserved=0>
>>> meaning you could write the "glue" what apache projects hortonworks had
>>> written to roll out a set of services. I guess this is how they got around
>>> being open source. I guess I'll start looking into making my own stack
>>> from scratch. If anyone already has one or know of one, please let me know.
>>>
>>>
>>>
>>> On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <ga...@linaro.org>
>>> wrote:
>>>
>>> Apache Bigtop now has mpack and works with Ambari. You could try that.
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Ganesh
>>>
>>>
>>>
>>> On Mon, May 11, 2020 at 9:15 AM Matt Andruff <ma...@gmail.com>
>>> wrote:
>>>
>>> Hey I was just poking around today trying to figure out if ambari still
>>> works to install a cluster.
>>>
>>>
>>>
>>> I recently was able to install a Cloudera Manager installation.
>>>
>>>
>>>
>>> I was about to start an Ambari installation but when I poke the repos
>>> they all seem to need a user/password.
>>>
>>>
>>>
>>> Is there a non-hortonworks(cloudera) hadoop that can be still be
>>> installed with Amabari?
>>>
>>>
>>>
>>> If there is documentation somewhere just let me know, I couldn't find
>>> the answer googling as Hortonworks(cloudera) docs are all that come up.
>>>
>>>
>>>
>>> --
>>>
>>> Live life
>>> Laugh Often
>>>
>>>
>>>
>>> --
>>>
>>> IRC: ganeshraju@#linaro on irc.freenode.ne
>>> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0>
>>> t
>>>
>>>
>>>
>>>
>>> --
>>>
>>> Live life
>>> Laugh Often
>>>
>>>
>>>
>>> --
>>>
>>> IRC: ganeshraju@#linaro on irc.freenode.ne
>>> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0>
>>> t
>>>
>>>
>>>
>>> --
>>>
>>> IRC: ganeshraju@#linaro on irc.freenode.ne
>>> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0>
>>> t
>>>
>>>
>>
>> --
>> Live life
>> Laugh Often
>>
>
--
Live life
Laugh Often
Re: life after Hortonworks
Posted by Aaron Bossert <aa...@punchcyber.com>.
That would be greatly appreciated, @jeremy montgomery<ma...@gmail.com> ! And for a design document, is there an example floating around that would be appropriately formatted ?
Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: jeremy montgomery <al...@gmail.com>
Sent: Thursday, May 14, 2020 9:22:23 AM
To: user@ambari.apache.org <us...@ambari.apache.org>
Subject: Re: life after Hortonworks
@aaron - i don't think that the server/agent piece needs to be messed with. It is more puppet than ansible, but it works pretty well. It has a few issues that need to be worked out, but nothing insurmountable.
Agent/Server (pieces that need work)
* Ability to invalidate the agent cache.
* Multithreading - Currently alerts/restarts work on a concept of threads and timeouts. You can end up with too many concurrent tasks/alerts and things get backlogged waiting on the timeouts. However, if you up the threads you run into a lot of race conditions with auto-restart and alerts firing concurrently. ----> main piece here is it needs a prioritized queuing system locally.
* Order of restarts and hierarchy - Right now this is hard coded at the server level within a component and somewhat for full restarts based on the order on the left hand bar.
* Need the ability to modify the full restart sequence
* Need the ability to state whether a restart is blocking, blocking only when component is on the same server, and non-blocking.
* Need the ability to chain 2 different components together. Example, if i restart a kerberized hiveserver2 i can have it restart hue as well after it is restarted. Otherwise hue will error out since the spnego hashes are different.
I'll make some edits to the confluence page explaining how the stacks/mpacks/extensions work.
On Tue, May 12, 2020 at 10:51 AM Matt Andruff <ma...@gmail.com>> wrote:
@Aaron Bossert - For that type of change (Ansible) you probably need to write up a design paper. (I have seen this done, and I'm not sure it's required but seems to be common practice) Sounds like ansible would be a good fit.
@Jeremy - thanks for the insights, I might ping you if I get stuck trying to get into it. I certainly wish there was a clean standard for installation locations.
On Tue, May 12, 2020 at 10:36 AM Aaron Bossert <aa...@punchcyber.com>> wrote:
Jeremy,
I would be happy to tackle this with you and whoever else is willing…that being said, I have always had a love/hate relationship with Ambari…the stacks have always been intimidating to me….you mention revisiting the architecture….do you have any ideas for what might be better? In my opinion, I have always wanted to replace the guts of ambari with a system that is underpinned by Ansible…seems much more user-friendly and MUCH easier to configure and extend for those who are not comfortable with doing their own Java to Python and creating RPM’s and the like….What are your thoughts?
-- M. Aaron Bossert
From: jeremy montgomery <al...@gmail.com>>
Reply-To: "user@ambari.apache.org<ma...@ambari.apache.org>" <us...@ambari.apache.org>>
Date: Tuesday, May 12, 2020 at 9:14 AM
To: "user@ambari.apache.org<ma...@ambari.apache.org>" <us...@ambari.apache.org>>
Subject: Re: life after Hortonworks
I make mpacks all the time, but tend to push them as extensions because the stack architecture is byzantine.
* It is java that deploys python code. As such, the structure behaves like java, which makes it a pain in the rear to track down simple python functions. Sometimes the code ends up in common-services, sometimes it stays in the current version of stacks. Some stack components only use code in their latest version, some have code files strung through the folders of 8 previous versions and 3 stacks (looking at you LLAP).
* The install code doesn't know where it wants to be. sometimes its in the stack, sometimes it is in ambari. This means that a version of ambari tends to be hard coded to 1-2 versions of a stack.
* Changing javascript for a stack isn't possible. This means that stack components are hard coded into the ember.js with a bunch of if statements.
* Making additions to a stack feature is a major process (like adding hbase thrift to an existing hbase installation)
* Upgrades require a yum file regardless of installation method. so learning how to create dummy rpms is necessary.
I'd be willing to chip in and maintain the stack python code but the stack architecture really needs to be revisited.
On Mon, May 11, 2020 at 1:50 PM Aaron Bossert <aa...@punchcyber.com>> wrote:
Matt,
Yeah, my thought was to start with whatever the most recent HDP/HDF stack definition as a starting point. It just so happens that I have a backburner project to do this already. I have been using Hortonworks for a long time, but have found recently that I needed to install newer versions of Apache Druid and Apache Storm, which would require me to do a new stack…Full disclosure: I have NEVER mucked with stacks and am not a Python guy…I write in Scala/Java predominantly….that being said, I would be happy to collaborate on this if anyone feels that this would be worthwhile and useful to the broader community.
-- M. Aaron Bossert
From: Matt Andruff <ma...@gmail.com>>
Reply-To: "user@ambari.apache.org<ma...@ambari.apache.org>" <us...@ambari.apache.org>>
Date: Monday, May 11, 2020 at 1:46 PM
To: "user@ambari.apache.org<ma...@ambari.apache.org>" <us...@ambari.apache.org>>
Subject: Re: life after Hortonworks
Cloudera no longer uses ambari. They stuck with Cloudera manager in their release of CDP.(CDH+HDP=CDP)
https://docs.cloudera.com/cdpdc/7.0/overview/topics/cdpdc-overview.html<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.cloudera.com%2Fcdpdc%2F7.0%2Foverview%2Ftopics%2Fcdpdc-overview.html&data=01%7C01%7Caaron%40punchcyber.com%7C716267a58ddf4dfc2fa108d7f809e488%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=fWhpx7aJiGmBCl9J%2BaFOA0aNDILgfau9QHqi0NMt1dA%3D&reserved=0>
I don't think this means that Ambari is dead. I do think it means as stated that the community will need to take on packaging a stack and building rpms.(or at least packaging the stack)
The legacy code for stack of HDP 2.6 stream is out there already in the repo so it's just some work to create rpms of whatever Ambari wants to release.
I assume the stacks section is so poorly documented is because Hortonworks was doing the work of packaging. I'm not sure the level of effort needed to make a stack work but it seems like we could start with the last HDP build (3.1.4) and keep moving forward.
On Mon., May 11, 2020, 13:08 Stephen Boesch, <ja...@gmail.com>> wrote:
I am reading between the lines that ambari is no longer a strategic platform. Would someone please provide a link/reference to a Cloudera press release or blog describing this and maybe related decisions/roadmaps? thx!
Am Mo., 11. Mai 2020 um 10:05 Uhr schrieb Aaron Bossert <aa...@punchcyber.com>>:
For what it is worth, I have written blueprints before, but never stacks. The documentation and tutorials for ambari stacks and blueprints are horribly out of date, incomplete, or flat out missing. Perhaps that could be an initial task for the community to undertake so that those of us who were using the Hortonworks suite of tools and were comfortable with Ambari can sever the cord, as it were....relying on commercial companies to support open source tools once their objectives have changed is rarely a good thing.
Get Outlook for iOS<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=01%7C01%7Caaron%40punchcyber.com%7C716267a58ddf4dfc2fa108d7f809e488%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=VoU2w9zFujl7NkXXLAgCa8NQ9T0%2FvU3FjCVrbCgZfCg%3D&reserved=0>
________________________________
From: Ganesh Raju <ga...@linaro.org>>
Sent: Monday, May 11, 2020 12:57:50 PM
To: user@ambari.apache.org<ma...@ambari.apache.org> <us...@ambari.apache.org>>
Subject: Re: life after Hortonworks
Here is more Apache Bigtop info
releases<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FReleases&data=01%7C01%7Caaron%40punchcyber.com%7C716267a58ddf4dfc2fa108d7f809e488%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=jysf1NLM%2FOnrQn%2F%2BkWhBsJKb9rE2mJZN3Il52%2Bp7XOo%3D&reserved=0>
mailinglist<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbigtop.apache.org%2Fmail-lists.html&data=01%7C01%7Caaron%40punchcyber.com%7C716267a58ddf4dfc2fa108d7f809e488%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=xW%2FiIQXmke6yYq13k4IFMzeUNaq7%2Bzm%2BbNEB9q5Gyec%3D&reserved=0>
list of components<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FBigtop%2B1.4.0%2BRelease&data=01%7C01%7Caaron%40punchcyber.com%7C716267a58ddf4dfc2fa108d7f809e488%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=l36DcnEkNgENsmmltQQzZ9cpS9dRlHp53fQd7%2FCoKwo%3D&reserved=0>
Thanks
Ganesh Raju
On Mon, May 11, 2020 at 10:00 AM Ganesh Raju <ga...@linaro.org>> wrote:
Yes that's what Bigtop did. BTW, Apache Bigtop already has hadoop and 30 other components.
Ganesh
On Mon, May 11, 2020 at 9:56 AM Matt Andruff <ma...@gmail.com>> wrote:
I kept digging and I found that ambari supports stacks (that we have all seen before). You can write your own custom stack<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAMBARI%2FOverview%23Overview-Structure&data=01%7C01%7Caaron%40punchcyber.com%7C716267a58ddf4dfc2fa108d7f809e488%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=3U4gbUI%2FDhPPjkYti6%2FH6ooOBg8z3rizHXG%2F3F2%2BnGA%3D&reserved=0> meaning you could write the "glue" what apache projects hortonworks had written to roll out a set of services. I guess this is how they got around being open source. I guess I'll start looking into making my own stack from scratch. If anyone already has one or know of one, please let me know.
On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <ga...@linaro.org>> wrote:
Apache Bigtop now has mpack and works with Ambari. You could try that.
Thanks,
Ganesh
On Mon, May 11, 2020 at 9:15 AM Matt Andruff <ma...@gmail.com>> wrote:
Hey I was just poking around today trying to figure out if ambari still works to install a cluster.
I recently was able to install a Cloudera Manager installation.
I was about to start an Ambari installation but when I poke the repos they all seem to need a user/password.
Is there a non-hortonworks(cloudera) hadoop that can be still be installed with Amabari?
If there is documentation somewhere just let me know, I couldn't find the answer googling as Hortonworks(cloudera) docs are all that come up.
--
Live life
Laugh Often
--
IRC: ganeshraju@#linaro on irc.freenode.ne<https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C716267a58ddf4dfc2fa108d7f809e488%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=%2BUJvTSdEGBmyT9XmYRza0KKKECDz0MIY8Nuzks%2FGW6g%3D&reserved=0>t
--
Live life
Laugh Often
--
IRC: ganeshraju@#linaro on irc.freenode.ne<https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C716267a58ddf4dfc2fa108d7f809e488%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=%2BUJvTSdEGBmyT9XmYRza0KKKECDz0MIY8Nuzks%2FGW6g%3D&reserved=0>t
--
IRC: ganeshraju@#linaro on irc.freenode.ne<https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C716267a58ddf4dfc2fa108d7f809e488%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=%2BUJvTSdEGBmyT9XmYRza0KKKECDz0MIY8Nuzks%2FGW6g%3D&reserved=0>t
--
Live life
Laugh Often
Re: life after Hortonworks
Posted by jeremy montgomery <al...@gmail.com>.
@aaron - i don't think that the server/agent piece needs to be messed
with. It is more puppet than ansible, but it works pretty well. It has a
few issues that need to be worked out, but nothing insurmountable.
Agent/Server (pieces that need work)
- Ability to invalidate the agent cache.
- Multithreading - Currently alerts/restarts work on a concept of
threads and timeouts. You can end up with too many concurrent tasks/alerts
and things get backlogged waiting on the timeouts. However, if you up the
threads you run into a lot of race conditions with auto-restart and alerts
firing concurrently. ----> main piece here is it needs a prioritized
queuing system locally.
- Order of restarts and hierarchy - Right now this is hard coded at the
server level within a component and somewhat for full restarts based on the
order on the left hand bar.
- Need the ability to modify the full restart sequence
- Need the ability to state whether a restart is blocking, blocking
only when component is on the same server, and non-blocking.
- Need the ability to chain 2 different components together.
Example, if i restart a kerberized hiveserver2 i can have it
restart hue as
well after it is restarted. Otherwise hue will error out since
the spnego
hashes are different.
I'll make some edits to the confluence page explaining how the
stacks/mpacks/extensions work.
On Tue, May 12, 2020 at 10:51 AM Matt Andruff <ma...@gmail.com>
wrote:
> @Aaron Bossert - For that type of change (Ansible) you probably need to
> write up a design paper. (I have seen this done, and I'm not sure it's
> required but seems to be common practice) Sounds like ansible would be a
> good fit.
>
> @Jeremy - thanks for the insights, I might ping you if I get stuck trying
> to get into it. I certainly wish there was a clean standard for
> installation locations.
>
>
>
> On Tue, May 12, 2020 at 10:36 AM Aaron Bossert <aa...@punchcyber.com>
> wrote:
>
>> Jeremy,
>>
>>
>>
>> I would be happy to tackle this with you and whoever else is willing…that
>> being said, I have always had a love/hate relationship with Ambari…the
>> stacks have always been intimidating to me….you mention revisiting the
>> architecture….do you have any ideas for what might be better? In my
>> opinion, I have always wanted to replace the guts of ambari with a system
>> that is underpinned by Ansible…seems much more user-friendly and MUCH
>> easier to configure and extend for those who are not comfortable with doing
>> their own Java to Python and creating RPM’s and the like….What are your
>> thoughts?
>>
>>
>>
>>
>>
>> -- M. Aaron Bossert
>>
>>
>>
>> *From: *jeremy montgomery <al...@gmail.com>
>> *Reply-To: *"user@ambari.apache.org" <us...@ambari.apache.org>
>> *Date: *Tuesday, May 12, 2020 at 9:14 AM
>> *To: *"user@ambari.apache.org" <us...@ambari.apache.org>
>> *Subject: *Re: life after Hortonworks
>>
>>
>>
>> I make mpacks all the time, but tend to push them as extensions because
>> the stack architecture is byzantine.
>>
>> - It is java that deploys python code. As such, the structure
>> behaves like java, which makes it a pain in the rear to track down simple
>> python functions. Sometimes the code ends up in common-services, sometimes
>> it stays in the current version of stacks. Some stack components only use
>> code in their latest version, some have code files strung through the
>> folders of 8 previous versions and 3 stacks (looking at you LLAP).
>> - The install code doesn't know where it wants to be. sometimes its
>> in the stack, sometimes it is in ambari. This means that a version of
>> ambari tends to be hard coded to 1-2 versions of a stack.
>> - Changing javascript for a stack isn't possible. This means that
>> stack components are hard coded into the ember.js with a bunch of if
>> statements.
>> - Making additions to a stack feature is a major process (like adding
>> hbase thrift to an existing hbase installation)
>> - Upgrades require a yum file regardless of installation method. so
>> learning how to create dummy rpms is necessary.
>>
>> I'd be willing to chip in and maintain the stack python code but the
>> stack architecture really needs to be revisited.
>>
>>
>>
>> On Mon, May 11, 2020 at 1:50 PM Aaron Bossert <aa...@punchcyber.com>
>> wrote:
>>
>> Matt,
>>
>>
>>
>> Yeah, my thought was to start with whatever the most recent HDP/HDF stack
>> definition as a starting point. It just so happens that I have a
>> backburner project to do this already. I have been using Hortonworks for a
>> long time, but have found recently that I needed to install newer versions
>> of Apache Druid and Apache Storm, which would require me to do a new
>> stack…Full disclosure: I have NEVER mucked with stacks and am not a Python
>> guy…I write in Scala/Java predominantly….that being said, I would be happy
>> to collaborate on this if anyone feels that this would be worthwhile and
>> useful to the broader community.
>>
>>
>>
>> -- M. Aaron Bossert
>>
>>
>>
>> *From: *Matt Andruff <ma...@gmail.com>
>> *Reply-To: *"user@ambari.apache.org" <us...@ambari.apache.org>
>> *Date: *Monday, May 11, 2020 at 1:46 PM
>> *To: *"user@ambari.apache.org" <us...@ambari.apache.org>
>> *Subject: *Re: life after Hortonworks
>>
>>
>>
>> Cloudera no longer uses ambari. They stuck with Cloudera manager in their
>> release of CDP.(CDH+HDP=CDP)
>>
>>
>>
>> https://docs.cloudera.com/cdpdc/7.0/overview/topics/cdpdc-overview.html
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.cloudera.com%2Fcdpdc%2F7.0%2Foverview%2Ftopics%2Fcdpdc-overview.html&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=7%2BMaRSRcCIjjKVT777pp18IYYsXEAGk3VPdNuOexV2w%3D&reserved=0>
>>
>>
>>
>> I don't think this means that Ambari is dead. I do think it means as
>> stated that the community will need to take on packaging a stack and
>> building rpms.(or at least packaging the stack)
>>
>>
>>
>> The legacy code for stack of HDP 2.6 stream is out there already in the
>> repo so it's just some work to create rpms of whatever Ambari wants to
>> release.
>>
>>
>>
>> I assume the stacks section is so poorly documented is because
>> Hortonworks was doing the work of packaging. I'm not sure the level of
>> effort needed to make a stack work but it seems like we could start with
>> the last HDP build (3.1.4) and keep moving forward.
>>
>>
>>
>>
>>
>> On Mon., May 11, 2020, 13:08 Stephen Boesch, <ja...@gmail.com> wrote:
>>
>> I am reading between the lines that ambari is no longer a strategic
>> platform. Would someone please provide a link/reference to a Cloudera press
>> release or blog describing this and maybe related decisions/roadmaps? thx!
>>
>>
>>
>> Am Mo., 11. Mai 2020 um 10:05 Uhr schrieb Aaron Bossert <
>> aaron@punchcyber.com>:
>>
>> For what it is worth, I have written blueprints before, but never
>> stacks. The documentation and tutorials for ambari stacks and blueprints
>> are horribly out of date, incomplete, or flat out missing. Perhaps that
>> could be an initial task for the community to undertake so that those of us
>> who were using the Hortonworks suite of tools and were comfortable with
>> Ambari can sever the cord, as it were....relying on commercial companies to
>> support open source tools once their objectives have changed is rarely a
>> good thing.
>>
>>
>>
>> Get Outlook for iOS
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=AqkexLKW6pLpkhG0auiGnGpIy5aZpcc%2B%2BG53O2qCNek%3D&reserved=0>
>> ------------------------------
>>
>> *From:* Ganesh Raju <ga...@linaro.org>
>> *Sent:* Monday, May 11, 2020 12:57:50 PM
>> *To:* user@ambari.apache.org <us...@ambari.apache.org>
>> *Subject:* Re: life after Hortonworks
>>
>>
>>
>> Here is more Apache Bigtop info
>>
>>
>>
>> releases
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FReleases&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=8xGs5j8Akzib7cRVlEgWq8ORcz8dWduDXqu8ev69mTE%3D&reserved=0>
>>
>> mailinglist
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbigtop.apache.org%2Fmail-lists.html&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=NgLr8gAPF2BXSr%2FXnUuaR2%2FdDU0c40RlyCoxiATR%2FWw%3D&reserved=0>
>>
>> list of components
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FBigtop%2B1.4.0%2BRelease&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=vXMOynon6RZFabIVAo5PQ4D%2B6cB9rq%2BHSBbYBnH9uPI%3D&reserved=0>
>>
>>
>>
>> Thanks
>>
>> Ganesh Raju
>>
>>
>>
>>
>>
>> On Mon, May 11, 2020 at 10:00 AM Ganesh Raju <ga...@linaro.org>
>> wrote:
>>
>> Yes that's what Bigtop did. BTW, Apache Bigtop already has hadoop and 30
>> other components.
>>
>>
>>
>> Ganesh
>>
>>
>>
>> On Mon, May 11, 2020 at 9:56 AM Matt Andruff <ma...@gmail.com>
>> wrote:
>>
>> I kept digging and I found that ambari supports stacks (that we have all
>> seen before). You can write your own custom stack
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAMBARI%2FOverview%23Overview-Structure&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=3itBtwJy1mH3Eu2FbsVx0WurYZFlTlVY7VtV5D%2FbDjg%3D&reserved=0>
>> meaning you could write the "glue" what apache projects hortonworks had
>> written to roll out a set of services. I guess this is how they got around
>> being open source. I guess I'll start looking into making my own stack
>> from scratch. If anyone already has one or know of one, please let me know.
>>
>>
>>
>> On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <ga...@linaro.org>
>> wrote:
>>
>> Apache Bigtop now has mpack and works with Ambari. You could try that.
>>
>>
>>
>> Thanks,
>>
>> Ganesh
>>
>>
>>
>> On Mon, May 11, 2020 at 9:15 AM Matt Andruff <ma...@gmail.com>
>> wrote:
>>
>> Hey I was just poking around today trying to figure out if ambari still
>> works to install a cluster.
>>
>>
>>
>> I recently was able to install a Cloudera Manager installation.
>>
>>
>>
>> I was about to start an Ambari installation but when I poke the repos
>> they all seem to need a user/password.
>>
>>
>>
>> Is there a non-hortonworks(cloudera) hadoop that can be still be
>> installed with Amabari?
>>
>>
>>
>> If there is documentation somewhere just let me know, I couldn't find the
>> answer googling as Hortonworks(cloudera) docs are all that come up.
>>
>>
>>
>> --
>>
>> Live life
>> Laugh Often
>>
>>
>>
>> --
>>
>> IRC: ganeshraju@#linaro on irc.freenode.ne
>> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0>
>> t
>>
>>
>>
>>
>> --
>>
>> Live life
>> Laugh Often
>>
>>
>>
>> --
>>
>> IRC: ganeshraju@#linaro on irc.freenode.ne
>> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0>
>> t
>>
>>
>>
>> --
>>
>> IRC: ganeshraju@#linaro on irc.freenode.ne
>> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0>
>> t
>>
>>
>
> --
> Live life
> Laugh Often
>
Re: life after Hortonworks
Posted by Matt Andruff <ma...@gmail.com>.
@Aaron Bossert - For that type of change (Ansible) you probably need to
write up a design paper. (I have seen this done, and I'm not sure it's
required but seems to be common practice) Sounds like ansible would be a
good fit.
@Jeremy - thanks for the insights, I might ping you if I get stuck trying
to get into it. I certainly wish there was a clean standard for
installation locations.
On Tue, May 12, 2020 at 10:36 AM Aaron Bossert <aa...@punchcyber.com> wrote:
> Jeremy,
>
>
>
> I would be happy to tackle this with you and whoever else is willing…that
> being said, I have always had a love/hate relationship with Ambari…the
> stacks have always been intimidating to me….you mention revisiting the
> architecture….do you have any ideas for what might be better? In my
> opinion, I have always wanted to replace the guts of ambari with a system
> that is underpinned by Ansible…seems much more user-friendly and MUCH
> easier to configure and extend for those who are not comfortable with doing
> their own Java to Python and creating RPM’s and the like….What are your
> thoughts?
>
>
>
>
>
> -- M. Aaron Bossert
>
>
>
> *From: *jeremy montgomery <al...@gmail.com>
> *Reply-To: *"user@ambari.apache.org" <us...@ambari.apache.org>
> *Date: *Tuesday, May 12, 2020 at 9:14 AM
> *To: *"user@ambari.apache.org" <us...@ambari.apache.org>
> *Subject: *Re: life after Hortonworks
>
>
>
> I make mpacks all the time, but tend to push them as extensions because
> the stack architecture is byzantine.
>
> - It is java that deploys python code. As such, the structure behaves
> like java, which makes it a pain in the rear to track down simple python
> functions. Sometimes the code ends up in common-services, sometimes it
> stays in the current version of stacks. Some stack components only use
> code in their latest version, some have code files strung through the
> folders of 8 previous versions and 3 stacks (looking at you LLAP).
> - The install code doesn't know where it wants to be. sometimes its
> in the stack, sometimes it is in ambari. This means that a version of
> ambari tends to be hard coded to 1-2 versions of a stack.
> - Changing javascript for a stack isn't possible. This means that
> stack components are hard coded into the ember.js with a bunch of if
> statements.
> - Making additions to a stack feature is a major process (like adding
> hbase thrift to an existing hbase installation)
> - Upgrades require a yum file regardless of installation method. so
> learning how to create dummy rpms is necessary.
>
> I'd be willing to chip in and maintain the stack python code but the stack
> architecture really needs to be revisited.
>
>
>
> On Mon, May 11, 2020 at 1:50 PM Aaron Bossert <aa...@punchcyber.com>
> wrote:
>
> Matt,
>
>
>
> Yeah, my thought was to start with whatever the most recent HDP/HDF stack
> definition as a starting point. It just so happens that I have a
> backburner project to do this already. I have been using Hortonworks for a
> long time, but have found recently that I needed to install newer versions
> of Apache Druid and Apache Storm, which would require me to do a new
> stack…Full disclosure: I have NEVER mucked with stacks and am not a Python
> guy…I write in Scala/Java predominantly….that being said, I would be happy
> to collaborate on this if anyone feels that this would be worthwhile and
> useful to the broader community.
>
>
>
> -- M. Aaron Bossert
>
>
>
> *From: *Matt Andruff <ma...@gmail.com>
> *Reply-To: *"user@ambari.apache.org" <us...@ambari.apache.org>
> *Date: *Monday, May 11, 2020 at 1:46 PM
> *To: *"user@ambari.apache.org" <us...@ambari.apache.org>
> *Subject: *Re: life after Hortonworks
>
>
>
> Cloudera no longer uses ambari. They stuck with Cloudera manager in their
> release of CDP.(CDH+HDP=CDP)
>
>
>
> https://docs.cloudera.com/cdpdc/7.0/overview/topics/cdpdc-overview.html
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.cloudera.com%2Fcdpdc%2F7.0%2Foverview%2Ftopics%2Fcdpdc-overview.html&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=7%2BMaRSRcCIjjKVT777pp18IYYsXEAGk3VPdNuOexV2w%3D&reserved=0>
>
>
>
> I don't think this means that Ambari is dead. I do think it means as
> stated that the community will need to take on packaging a stack and
> building rpms.(or at least packaging the stack)
>
>
>
> The legacy code for stack of HDP 2.6 stream is out there already in the
> repo so it's just some work to create rpms of whatever Ambari wants to
> release.
>
>
>
> I assume the stacks section is so poorly documented is because Hortonworks
> was doing the work of packaging. I'm not sure the level of effort needed
> to make a stack work but it seems like we could start with the last HDP
> build (3.1.4) and keep moving forward.
>
>
>
>
>
> On Mon., May 11, 2020, 13:08 Stephen Boesch, <ja...@gmail.com> wrote:
>
> I am reading between the lines that ambari is no longer a strategic
> platform. Would someone please provide a link/reference to a Cloudera press
> release or blog describing this and maybe related decisions/roadmaps? thx!
>
>
>
> Am Mo., 11. Mai 2020 um 10:05 Uhr schrieb Aaron Bossert <
> aaron@punchcyber.com>:
>
> For what it is worth, I have written blueprints before, but never stacks.
> The documentation and tutorials for ambari stacks and blueprints are
> horribly out of date, incomplete, or flat out missing. Perhaps that could
> be an initial task for the community to undertake so that those of us who
> were using the Hortonworks suite of tools and were comfortable with Ambari
> can sever the cord, as it were....relying on commercial companies to
> support open source tools once their objectives have changed is rarely a
> good thing.
>
>
>
> Get Outlook for iOS
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=AqkexLKW6pLpkhG0auiGnGpIy5aZpcc%2B%2BG53O2qCNek%3D&reserved=0>
> ------------------------------
>
> *From:* Ganesh Raju <ga...@linaro.org>
> *Sent:* Monday, May 11, 2020 12:57:50 PM
> *To:* user@ambari.apache.org <us...@ambari.apache.org>
> *Subject:* Re: life after Hortonworks
>
>
>
> Here is more Apache Bigtop info
>
>
>
> releases
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FReleases&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=8xGs5j8Akzib7cRVlEgWq8ORcz8dWduDXqu8ev69mTE%3D&reserved=0>
>
> mailinglist
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbigtop.apache.org%2Fmail-lists.html&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=NgLr8gAPF2BXSr%2FXnUuaR2%2FdDU0c40RlyCoxiATR%2FWw%3D&reserved=0>
>
> list of components
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FBigtop%2B1.4.0%2BRelease&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=vXMOynon6RZFabIVAo5PQ4D%2B6cB9rq%2BHSBbYBnH9uPI%3D&reserved=0>
>
>
>
> Thanks
>
> Ganesh Raju
>
>
>
>
>
> On Mon, May 11, 2020 at 10:00 AM Ganesh Raju <ga...@linaro.org>
> wrote:
>
> Yes that's what Bigtop did. BTW, Apache Bigtop already has hadoop and 30
> other components.
>
>
>
> Ganesh
>
>
>
> On Mon, May 11, 2020 at 9:56 AM Matt Andruff <ma...@gmail.com>
> wrote:
>
> I kept digging and I found that ambari supports stacks (that we have all
> seen before). You can write your own custom stack
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAMBARI%2FOverview%23Overview-Structure&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=3itBtwJy1mH3Eu2FbsVx0WurYZFlTlVY7VtV5D%2FbDjg%3D&reserved=0>
> meaning you could write the "glue" what apache projects hortonworks had
> written to roll out a set of services. I guess this is how they got around
> being open source. I guess I'll start looking into making my own stack
> from scratch. If anyone already has one or know of one, please let me know.
>
>
>
> On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <ga...@linaro.org>
> wrote:
>
> Apache Bigtop now has mpack and works with Ambari. You could try that.
>
>
>
> Thanks,
>
> Ganesh
>
>
>
> On Mon, May 11, 2020 at 9:15 AM Matt Andruff <ma...@gmail.com>
> wrote:
>
> Hey I was just poking around today trying to figure out if ambari still
> works to install a cluster.
>
>
>
> I recently was able to install a Cloudera Manager installation.
>
>
>
> I was about to start an Ambari installation but when I poke the repos they
> all seem to need a user/password.
>
>
>
> Is there a non-hortonworks(cloudera) hadoop that can be still be installed
> with Amabari?
>
>
>
> If there is documentation somewhere just let me know, I couldn't find the
> answer googling as Hortonworks(cloudera) docs are all that come up.
>
>
>
> --
>
> Live life
> Laugh Often
>
>
>
> --
>
> IRC: ganeshraju@#linaro on irc.freenode.ne
> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0>
> t
>
>
>
>
> --
>
> Live life
> Laugh Often
>
>
>
> --
>
> IRC: ganeshraju@#linaro on irc.freenode.ne
> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0>
> t
>
>
>
> --
>
> IRC: ganeshraju@#linaro on irc.freenode.ne
> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0>
> t
>
>
--
Live life
Laugh Often
Re: life after Hortonworks
Posted by Aaron Bossert <aa...@punchcyber.com>.
Jeremy,
I would be happy to tackle this with you and whoever else is willing…that being said, I have always had a love/hate relationship with Ambari…the stacks have always been intimidating to me….you mention revisiting the architecture….do you have any ideas for what might be better? In my opinion, I have always wanted to replace the guts of ambari with a system that is underpinned by Ansible…seems much more user-friendly and MUCH easier to configure and extend for those who are not comfortable with doing their own Java to Python and creating RPM’s and the like….What are your thoughts?
-- M. Aaron Bossert
From: jeremy montgomery <al...@gmail.com>
Reply-To: "user@ambari.apache.org" <us...@ambari.apache.org>
Date: Tuesday, May 12, 2020 at 9:14 AM
To: "user@ambari.apache.org" <us...@ambari.apache.org>
Subject: Re: life after Hortonworks
I make mpacks all the time, but tend to push them as extensions because the stack architecture is byzantine.
* It is java that deploys python code. As such, the structure behaves like java, which makes it a pain in the rear to track down simple python functions. Sometimes the code ends up in common-services, sometimes it stays in the current version of stacks. Some stack components only use code in their latest version, some have code files strung through the folders of 8 previous versions and 3 stacks (looking at you LLAP).
* The install code doesn't know where it wants to be. sometimes its in the stack, sometimes it is in ambari. This means that a version of ambari tends to be hard coded to 1-2 versions of a stack.
* Changing javascript for a stack isn't possible. This means that stack components are hard coded into the ember.js with a bunch of if statements.
* Making additions to a stack feature is a major process (like adding hbase thrift to an existing hbase installation)
* Upgrades require a yum file regardless of installation method. so learning how to create dummy rpms is necessary.
I'd be willing to chip in and maintain the stack python code but the stack architecture really needs to be revisited.
On Mon, May 11, 2020 at 1:50 PM Aaron Bossert <aa...@punchcyber.com>> wrote:
Matt,
Yeah, my thought was to start with whatever the most recent HDP/HDF stack definition as a starting point. It just so happens that I have a backburner project to do this already. I have been using Hortonworks for a long time, but have found recently that I needed to install newer versions of Apache Druid and Apache Storm, which would require me to do a new stack…Full disclosure: I have NEVER mucked with stacks and am not a Python guy…I write in Scala/Java predominantly….that being said, I would be happy to collaborate on this if anyone feels that this would be worthwhile and useful to the broader community.
-- M. Aaron Bossert
From: Matt Andruff <ma...@gmail.com>>
Reply-To: "user@ambari.apache.org<ma...@ambari.apache.org>" <us...@ambari.apache.org>>
Date: Monday, May 11, 2020 at 1:46 PM
To: "user@ambari.apache.org<ma...@ambari.apache.org>" <us...@ambari.apache.org>>
Subject: Re: life after Hortonworks
Cloudera no longer uses ambari. They stuck with Cloudera manager in their release of CDP.(CDH+HDP=CDP)
https://docs.cloudera.com/cdpdc/7.0/overview/topics/cdpdc-overview.html<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.cloudera.com%2Fcdpdc%2F7.0%2Foverview%2Ftopics%2Fcdpdc-overview.html&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=7%2BMaRSRcCIjjKVT777pp18IYYsXEAGk3VPdNuOexV2w%3D&reserved=0>
I don't think this means that Ambari is dead. I do think it means as stated that the community will need to take on packaging a stack and building rpms.(or at least packaging the stack)
The legacy code for stack of HDP 2.6 stream is out there already in the repo so it's just some work to create rpms of whatever Ambari wants to release.
I assume the stacks section is so poorly documented is because Hortonworks was doing the work of packaging. I'm not sure the level of effort needed to make a stack work but it seems like we could start with the last HDP build (3.1.4) and keep moving forward.
On Mon., May 11, 2020, 13:08 Stephen Boesch, <ja...@gmail.com>> wrote:
I am reading between the lines that ambari is no longer a strategic platform. Would someone please provide a link/reference to a Cloudera press release or blog describing this and maybe related decisions/roadmaps? thx!
Am Mo., 11. Mai 2020 um 10:05 Uhr schrieb Aaron Bossert <aa...@punchcyber.com>>:
For what it is worth, I have written blueprints before, but never stacks. The documentation and tutorials for ambari stacks and blueprints are horribly out of date, incomplete, or flat out missing. Perhaps that could be an initial task for the community to undertake so that those of us who were using the Hortonworks suite of tools and were comfortable with Ambari can sever the cord, as it were....relying on commercial companies to support open source tools once their objectives have changed is rarely a good thing.
Get Outlook for iOS<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=AqkexLKW6pLpkhG0auiGnGpIy5aZpcc%2B%2BG53O2qCNek%3D&reserved=0>
________________________________
From: Ganesh Raju <ga...@linaro.org>>
Sent: Monday, May 11, 2020 12:57:50 PM
To: user@ambari.apache.org<ma...@ambari.apache.org> <us...@ambari.apache.org>>
Subject: Re: life after Hortonworks
Here is more Apache Bigtop info
releases<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FReleases&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=8xGs5j8Akzib7cRVlEgWq8ORcz8dWduDXqu8ev69mTE%3D&reserved=0>
mailinglist<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbigtop.apache.org%2Fmail-lists.html&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=NgLr8gAPF2BXSr%2FXnUuaR2%2FdDU0c40RlyCoxiATR%2FWw%3D&reserved=0>
list of components<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FBigtop%2B1.4.0%2BRelease&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=vXMOynon6RZFabIVAo5PQ4D%2B6cB9rq%2BHSBbYBnH9uPI%3D&reserved=0>
Thanks
Ganesh Raju
On Mon, May 11, 2020 at 10:00 AM Ganesh Raju <ga...@linaro.org>> wrote:
Yes that's what Bigtop did. BTW, Apache Bigtop already has hadoop and 30 other components.
Ganesh
On Mon, May 11, 2020 at 9:56 AM Matt Andruff <ma...@gmail.com>> wrote:
I kept digging and I found that ambari supports stacks (that we have all seen before). You can write your own custom stack<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAMBARI%2FOverview%23Overview-Structure&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=3itBtwJy1mH3Eu2FbsVx0WurYZFlTlVY7VtV5D%2FbDjg%3D&reserved=0> meaning you could write the "glue" what apache projects hortonworks had written to roll out a set of services. I guess this is how they got around being open source. I guess I'll start looking into making my own stack from scratch. If anyone already has one or know of one, please let me know.
On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <ga...@linaro.org>> wrote:
Apache Bigtop now has mpack and works with Ambari. You could try that.
Thanks,
Ganesh
On Mon, May 11, 2020 at 9:15 AM Matt Andruff <ma...@gmail.com>> wrote:
Hey I was just poking around today trying to figure out if ambari still works to install a cluster.
I recently was able to install a Cloudera Manager installation.
I was about to start an Ambari installation but when I poke the repos they all seem to need a user/password.
Is there a non-hortonworks(cloudera) hadoop that can be still be installed with Amabari?
If there is documentation somewhere just let me know, I couldn't find the answer googling as Hortonworks(cloudera) docs are all that come up.
--
Live life
Laugh Often
--
IRC: ganeshraju@#linaro on irc.freenode.ne<https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0>t
--
Live life
Laugh Often
--
IRC: ganeshraju@#linaro on irc.freenode.ne<https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0>t
--
IRC: ganeshraju@#linaro on irc.freenode.ne<https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C89b759ffe21a47ee874a08d7f6766280%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=uOTbOwThQhNWnkTGmf3HZ6lIxPG0jjmxMUFOdANlN8I%3D&reserved=0>t
Re: life after Hortonworks
Posted by jeremy montgomery <al...@gmail.com>.
I make mpacks all the time, but tend to push them as extensions because the
stack architecture is byzantine.
- It is java that deploys python code. As such, the structure behaves
like java, which makes it a pain in the rear to track down simple python
functions. Sometimes the code ends up in common-services, sometimes it
stays in the current version of stacks. Some stack components only use
code in their latest version, some have code files strung through the
folders of 8 previous versions and 3 stacks (looking at you LLAP).
- The install code doesn't know where it wants to be. sometimes its in
the stack, sometimes it is in ambari. This means that a version of ambari
tends to be hard coded to 1-2 versions of a stack.
- Changing javascript for a stack isn't possible. This means that stack
components are hard coded into the ember.js with a bunch of if statements.
- Making additions to a stack feature is a major process (like adding
hbase thrift to an existing hbase installation)
- Upgrades require a yum file regardless of installation method. so
learning how to create dummy rpms is necessary.
I'd be willing to chip in and maintain the stack python code but the stack
architecture really needs to be revisited.
On Mon, May 11, 2020 at 1:50 PM Aaron Bossert <aa...@punchcyber.com> wrote:
> Matt,
>
>
>
> Yeah, my thought was to start with whatever the most recent HDP/HDF stack
> definition as a starting point. It just so happens that I have a
> backburner project to do this already. I have been using Hortonworks for a
> long time, but have found recently that I needed to install newer versions
> of Apache Druid and Apache Storm, which would require me to do a new
> stack…Full disclosure: I have NEVER mucked with stacks and am not a Python
> guy…I write in Scala/Java predominantly….that being said, I would be happy
> to collaborate on this if anyone feels that this would be worthwhile and
> useful to the broader community.
>
>
>
> -- M. Aaron Bossert
>
>
>
> *From: *Matt Andruff <ma...@gmail.com>
> *Reply-To: *"user@ambari.apache.org" <us...@ambari.apache.org>
> *Date: *Monday, May 11, 2020 at 1:46 PM
> *To: *"user@ambari.apache.org" <us...@ambari.apache.org>
> *Subject: *Re: life after Hortonworks
>
>
>
> Cloudera no longer uses ambari. They stuck with Cloudera manager in their
> release of CDP.(CDH+HDP=CDP)
>
>
>
> https://docs.cloudera.com/cdpdc/7.0/overview/topics/cdpdc-overview.html
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.cloudera.com%2Fcdpdc%2F7.0%2Foverview%2Ftopics%2Fcdpdc-overview.html&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=9t291neBjp4%2F2XnAvU34geDz9QgB%2BNAISHRyjvcvcog%3D&reserved=0>
>
>
>
> I don't think this means that Ambari is dead. I do think it means as
> stated that the community will need to take on packaging a stack and
> building rpms.(or at least packaging the stack)
>
>
>
> The legacy code for stack of HDP 2.6 stream is out there already in the
> repo so it's just some work to create rpms of whatever Ambari wants to
> release.
>
>
>
> I assume the stacks section is so poorly documented is because Hortonworks
> was doing the work of packaging. I'm not sure the level of effort needed
> to make a stack work but it seems like we could start with the last HDP
> build (3.1.4) and keep moving forward.
>
>
>
>
>
> On Mon., May 11, 2020, 13:08 Stephen Boesch, <ja...@gmail.com> wrote:
>
> I am reading between the lines that ambari is no longer a strategic
> platform. Would someone please provide a link/reference to a Cloudera press
> release or blog describing this and maybe related decisions/roadmaps? thx!
>
>
>
> Am Mo., 11. Mai 2020 um 10:05 Uhr schrieb Aaron Bossert <
> aaron@punchcyber.com>:
>
> For what it is worth, I have written blueprints before, but never stacks.
> The documentation and tutorials for ambari stacks and blueprints are
> horribly out of date, incomplete, or flat out missing. Perhaps that could
> be an initial task for the community to undertake so that those of us who
> were using the Hortonworks suite of tools and were comfortable with Ambari
> can sever the cord, as it were....relying on commercial companies to
> support open source tools once their objectives have changed is rarely a
> good thing.
>
>
>
> Get Outlook for iOS
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=7Y415fNQH3d4y67LbSIYGhLJxaCLWPGi6qcOutlSLGY%3D&reserved=0>
> ------------------------------
>
> *From:* Ganesh Raju <ga...@linaro.org>
> *Sent:* Monday, May 11, 2020 12:57:50 PM
> *To:* user@ambari.apache.org <us...@ambari.apache.org>
> *Subject:* Re: life after Hortonworks
>
>
>
> Here is more Apache Bigtop info
>
>
>
> releases
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FReleases&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=Ee0QKqwFQ%2FlBtha%2Bkw8enOtkr4PMD9PYWvCDyrvjqS0%3D&reserved=0>
>
> mailinglist
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbigtop.apache.org%2Fmail-lists.html&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=TcKPHYO%2Buhi%2BgLEXwjQtMWKb0nbuwinJUjiKDly2iVA%3D&reserved=0>
>
> list of components
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FBigtop%2B1.4.0%2BRelease&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=tMDI6ARheCagibWzd4liiGVgLruvIfTnVztQuaExVK0%3D&reserved=0>
>
>
>
> Thanks
>
> Ganesh Raju
>
>
>
>
>
> On Mon, May 11, 2020 at 10:00 AM Ganesh Raju <ga...@linaro.org>
> wrote:
>
> Yes that's what Bigtop did. BTW, Apache Bigtop already has hadoop and 30
> other components.
>
>
>
> Ganesh
>
>
>
> On Mon, May 11, 2020 at 9:56 AM Matt Andruff <ma...@gmail.com>
> wrote:
>
> I kept digging and I found that ambari supports stacks (that we have all
> seen before). You can write your own custom stack
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAMBARI%2FOverview%23Overview-Structure&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=4KdHTtYYw%2B9AN6jH0j8GN%2FTaG7kT57byTJph9Ss%2B5uw%3D&reserved=0>
> meaning you could write the "glue" what apache projects hortonworks had
> written to roll out a set of services. I guess this is how they got around
> being open source. I guess I'll start looking into making my own stack
> from scratch. If anyone already has one or know of one, please let me know.
>
>
>
> On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <ga...@linaro.org>
> wrote:
>
> Apache Bigtop now has mpack and works with Ambari. You could try that.
>
>
>
> Thanks,
>
> Ganesh
>
>
>
> On Mon, May 11, 2020 at 9:15 AM Matt Andruff <ma...@gmail.com>
> wrote:
>
> Hey I was just poking around today trying to figure out if ambari still
> works to install a cluster.
>
>
>
> I recently was able to install a Cloudera Manager installation.
>
>
>
> I was about to start an Ambari installation but when I poke the repos they
> all seem to need a user/password.
>
>
>
> Is there a non-hortonworks(cloudera) hadoop that can be still be installed
> with Amabari?
>
>
>
> If there is documentation somewhere just let me know, I couldn't find the
> answer googling as Hortonworks(cloudera) docs are all that come up.
>
>
>
> --
>
> Live life
> Laugh Often
>
>
>
> --
>
> IRC: ganeshraju@#linaro on irc.freenode.ne
> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=jBZn5YdHmsYjt5kToIBepOIu3BTJ3owQnn962F1JZDM%3D&reserved=0>
> t
>
>
>
>
> --
>
> Live life
> Laugh Often
>
>
>
> --
>
> IRC: ganeshraju@#linaro on irc.freenode.ne
> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=jBZn5YdHmsYjt5kToIBepOIu3BTJ3owQnn962F1JZDM%3D&reserved=0>
> t
>
>
>
> --
>
> IRC: ganeshraju@#linaro on irc.freenode.ne
> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=jBZn5YdHmsYjt5kToIBepOIu3BTJ3owQnn962F1JZDM%3D&reserved=0>
> t
>
>
Re: life after Hortonworks
Posted by Aaron Bossert <aa...@punchcyber.com>.
Matt,
Yeah, my thought was to start with whatever the most recent HDP/HDF stack definition as a starting point. It just so happens that I have a backburner project to do this already. I have been using Hortonworks for a long time, but have found recently that I needed to install newer versions of Apache Druid and Apache Storm, which would require me to do a new stack…Full disclosure: I have NEVER mucked with stacks and am not a Python guy…I write in Scala/Java predominantly….that being said, I would be happy to collaborate on this if anyone feels that this would be worthwhile and useful to the broader community.
-- M. Aaron Bossert
From: Matt Andruff <ma...@gmail.com>
Reply-To: "user@ambari.apache.org" <us...@ambari.apache.org>
Date: Monday, May 11, 2020 at 1:46 PM
To: "user@ambari.apache.org" <us...@ambari.apache.org>
Subject: Re: life after Hortonworks
Cloudera no longer uses ambari. They stuck with Cloudera manager in their release of CDP.(CDH+HDP=CDP)
https://docs.cloudera.com/cdpdc/7.0/overview/topics/cdpdc-overview.html<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.cloudera.com%2Fcdpdc%2F7.0%2Foverview%2Ftopics%2Fcdpdc-overview.html&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=9t291neBjp4%2F2XnAvU34geDz9QgB%2BNAISHRyjvcvcog%3D&reserved=0>
I don't think this means that Ambari is dead. I do think it means as stated that the community will need to take on packaging a stack and building rpms.(or at least packaging the stack)
The legacy code for stack of HDP 2.6 stream is out there already in the repo so it's just some work to create rpms of whatever Ambari wants to release.
I assume the stacks section is so poorly documented is because Hortonworks was doing the work of packaging. I'm not sure the level of effort needed to make a stack work but it seems like we could start with the last HDP build (3.1.4) and keep moving forward.
On Mon., May 11, 2020, 13:08 Stephen Boesch, <ja...@gmail.com>> wrote:
I am reading between the lines that ambari is no longer a strategic platform. Would someone please provide a link/reference to a Cloudera press release or blog describing this and maybe related decisions/roadmaps? thx!
Am Mo., 11. Mai 2020 um 10:05 Uhr schrieb Aaron Bossert <aa...@punchcyber.com>>:
For what it is worth, I have written blueprints before, but never stacks. The documentation and tutorials for ambari stacks and blueprints are horribly out of date, incomplete, or flat out missing. Perhaps that could be an initial task for the community to undertake so that those of us who were using the Hortonworks suite of tools and were comfortable with Ambari can sever the cord, as it were....relying on commercial companies to support open source tools once their objectives have changed is rarely a good thing.
Get Outlook for iOS<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=7Y415fNQH3d4y67LbSIYGhLJxaCLWPGi6qcOutlSLGY%3D&reserved=0>
________________________________
From: Ganesh Raju <ga...@linaro.org>>
Sent: Monday, May 11, 2020 12:57:50 PM
To: user@ambari.apache.org<ma...@ambari.apache.org> <us...@ambari.apache.org>>
Subject: Re: life after Hortonworks
Here is more Apache Bigtop info
releases<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FReleases&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=Ee0QKqwFQ%2FlBtha%2Bkw8enOtkr4PMD9PYWvCDyrvjqS0%3D&reserved=0>
mailinglist<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbigtop.apache.org%2Fmail-lists.html&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=TcKPHYO%2Buhi%2BgLEXwjQtMWKb0nbuwinJUjiKDly2iVA%3D&reserved=0>
list of components<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FBigtop%2B1.4.0%2BRelease&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=tMDI6ARheCagibWzd4liiGVgLruvIfTnVztQuaExVK0%3D&reserved=0>
Thanks
Ganesh Raju
On Mon, May 11, 2020 at 10:00 AM Ganesh Raju <ga...@linaro.org>> wrote:
Yes that's what Bigtop did. BTW, Apache Bigtop already has hadoop and 30 other components.
Ganesh
On Mon, May 11, 2020 at 9:56 AM Matt Andruff <ma...@gmail.com>> wrote:
I kept digging and I found that ambari supports stacks (that we have all seen before). You can write your own custom stack<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAMBARI%2FOverview%23Overview-Structure&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=4KdHTtYYw%2B9AN6jH0j8GN%2FTaG7kT57byTJph9Ss%2B5uw%3D&reserved=0> meaning you could write the "glue" what apache projects hortonworks had written to roll out a set of services. I guess this is how they got around being open source. I guess I'll start looking into making my own stack from scratch. If anyone already has one or know of one, please let me know.
On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <ga...@linaro.org>> wrote:
Apache Bigtop now has mpack and works with Ambari. You could try that.
Thanks,
Ganesh
On Mon, May 11, 2020 at 9:15 AM Matt Andruff <ma...@gmail.com>> wrote:
Hey I was just poking around today trying to figure out if ambari still works to install a cluster.
I recently was able to install a Cloudera Manager installation.
I was about to start an Ambari installation but when I poke the repos they all seem to need a user/password.
Is there a non-hortonworks(cloudera) hadoop that can be still be installed with Amabari?
If there is documentation somewhere just let me know, I couldn't find the answer googling as Hortonworks(cloudera) docs are all that come up.
--
Live life
Laugh Often
--
IRC: ganeshraju@#linaro on irc.freenode.ne<https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=jBZn5YdHmsYjt5kToIBepOIu3BTJ3owQnn962F1JZDM%3D&reserved=0>t
--
Live life
Laugh Often
--
IRC: ganeshraju@#linaro on irc.freenode.ne<https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=jBZn5YdHmsYjt5kToIBepOIu3BTJ3owQnn962F1JZDM%3D&reserved=0>t
--
IRC: ganeshraju@#linaro on irc.freenode.ne<https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C25e617782c824376770c08d7f5d32e42%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=jBZn5YdHmsYjt5kToIBepOIu3BTJ3owQnn962F1JZDM%3D&reserved=0>t
Re: life after Hortonworks
Posted by Matt Andruff <ma...@gmail.com>.
Cloudera no longer uses ambari. They stuck with Cloudera manager in their
release of CDP.(CDH+HDP=CDP)
https://docs.cloudera.com/cdpdc/7.0/overview/topics/cdpdc-overview.html
I don't think this means that Ambari is dead. I do think it means as
stated that the community will need to take on packaging a stack and
building rpms.(or at least packaging the stack)
The legacy code for stack of HDP 2.6 stream is out there already in the
repo so it's just some work to create rpms of whatever Ambari wants to
release.
I assume the stacks section is so poorly documented is because Hortonworks
was doing the work of packaging. I'm not sure the level of effort needed
to make a stack work but it seems like we could start with the last HDP
build (3.1.4) and keep moving forward.
On Mon., May 11, 2020, 13:08 Stephen Boesch, <ja...@gmail.com> wrote:
> I am reading between the lines that ambari is no longer a strategic
> platform. Would someone please provide a link/reference to a Cloudera press
> release or blog describing this and maybe related decisions/roadmaps? thx!
>
> Am Mo., 11. Mai 2020 um 10:05 Uhr schrieb Aaron Bossert <
> aaron@punchcyber.com>:
>
>> For what it is worth, I have written blueprints before, but never
>> stacks. The documentation and tutorials for ambari stacks and blueprints
>> are horribly out of date, incomplete, or flat out missing. Perhaps that
>> could be an initial task for the community to undertake so that those of us
>> who were using the Hortonworks suite of tools and were comfortable with
>> Ambari can sever the cord, as it were....relying on commercial companies to
>> support open source tools once their objectives have changed is rarely a
>> good thing.
>>
>> Get Outlook for iOS <https://aka.ms/o0ukef>
>> ------------------------------
>> *From:* Ganesh Raju <ga...@linaro.org>
>> *Sent:* Monday, May 11, 2020 12:57:50 PM
>> *To:* user@ambari.apache.org <us...@ambari.apache.org>
>> *Subject:* Re: life after Hortonworks
>>
>> Here is more Apache Bigtop info
>>
>> releases
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FReleases&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=QqZETOUT42xKoDSVwdOFSgIS9cyS%2BeEBH12hM7e5gdI%3D&reserved=0>
>> mailinglist
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbigtop.apache.org%2Fmail-lists.html&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=EyqeVHJozqXHtFh76b7KQtJ4Tl%2FKhsHaVN6EBiW3ev0%3D&reserved=0>
>> list of components
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FBigtop%2B1.4.0%2BRelease&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=WmuQu3t9PBrITRGZnFCAm7i8dB0sxuqF%2BWHkOql75xc%3D&reserved=0>
>>
>> Thanks
>> Ganesh Raju
>>
>>
>> On Mon, May 11, 2020 at 10:00 AM Ganesh Raju <ga...@linaro.org>
>> wrote:
>>
>> Yes that's what Bigtop did. BTW, Apache Bigtop already has hadoop and 30
>> other components.
>>
>> Ganesh
>>
>> On Mon, May 11, 2020 at 9:56 AM Matt Andruff <ma...@gmail.com>
>> wrote:
>>
>> I kept digging and I found that ambari supports stacks (that we have all
>> seen before). You can write your own custom stack
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAMBARI%2FOverview%23Overview-Structure&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=BKlsv87RZhCVj83Y7WAUw0Dydo9jiqRIsYRpN0owXZk%3D&reserved=0>
>> meaning you could write the "glue" what apache projects hortonworks had
>> written to roll out a set of services. I guess this is how they got around
>> being open source. I guess I'll start looking into making my own stack
>> from scratch. If anyone already has one or know of one, please let me know.
>>
>> On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <ga...@linaro.org>
>> wrote:
>>
>> Apache Bigtop now has mpack and works with Ambari. You could try that.
>>
>> Thanks,
>> Ganesh
>>
>> On Mon, May 11, 2020 at 9:15 AM Matt Andruff <ma...@gmail.com>
>> wrote:
>>
>> Hey I was just poking around today trying to figure out if ambari still
>> works to install a cluster.
>>
>> I recently was able to install a Cloudera Manager installation.
>>
>> I was about to start an Ambari installation but when I poke the repos
>> they all seem to need a user/password.
>>
>> Is there a non-hortonworks(cloudera) hadoop that can be still be
>> installed with Amabari?
>>
>> If there is documentation somewhere just let me know, I couldn't find the
>> answer googling as Hortonworks(cloudera) docs are all that come up.
>>
>> --
>> Live life
>> Laugh Often
>>
>>
>>
>> --
>> IRC: ganeshraju@#linaro on irc.freenode.ne
>> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=78S3EpIVAqTe6BbaRCxjYMtghzttYoie%2FLHwYbFYCbc%3D&reserved=0>
>> t
>>
>>
>>
>> --
>> Live life
>> Laugh Often
>>
>>
>>
>> --
>> IRC: ganeshraju@#linaro on irc.freenode.ne
>> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=78S3EpIVAqTe6BbaRCxjYMtghzttYoie%2FLHwYbFYCbc%3D&reserved=0>
>> t
>>
>>
>>
>> --
>> IRC: ganeshraju@#linaro on irc.freenode.ne
>> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=78S3EpIVAqTe6BbaRCxjYMtghzttYoie%2FLHwYbFYCbc%3D&reserved=0>
>> t
>>
>
Re: life after Hortonworks
Posted by Stephen Boesch <ja...@gmail.com>.
I am reading between the lines that ambari is no longer a strategic
platform. Would someone please provide a link/reference to a Cloudera press
release or blog describing this and maybe related decisions/roadmaps? thx!
Am Mo., 11. Mai 2020 um 10:05 Uhr schrieb Aaron Bossert <
aaron@punchcyber.com>:
> For what it is worth, I have written blueprints before, but never stacks.
> The documentation and tutorials for ambari stacks and blueprints are
> horribly out of date, incomplete, or flat out missing. Perhaps that could
> be an initial task for the community to undertake so that those of us who
> were using the Hortonworks suite of tools and were comfortable with Ambari
> can sever the cord, as it were....relying on commercial companies to
> support open source tools once their objectives have changed is rarely a
> good thing.
>
> Get Outlook for iOS <https://aka.ms/o0ukef>
> ------------------------------
> *From:* Ganesh Raju <ga...@linaro.org>
> *Sent:* Monday, May 11, 2020 12:57:50 PM
> *To:* user@ambari.apache.org <us...@ambari.apache.org>
> *Subject:* Re: life after Hortonworks
>
> Here is more Apache Bigtop info
>
> releases
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FReleases&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=QqZETOUT42xKoDSVwdOFSgIS9cyS%2BeEBH12hM7e5gdI%3D&reserved=0>
> mailinglist
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbigtop.apache.org%2Fmail-lists.html&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=EyqeVHJozqXHtFh76b7KQtJ4Tl%2FKhsHaVN6EBiW3ev0%3D&reserved=0>
> list of components
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FBigtop%2B1.4.0%2BRelease&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=WmuQu3t9PBrITRGZnFCAm7i8dB0sxuqF%2BWHkOql75xc%3D&reserved=0>
>
> Thanks
> Ganesh Raju
>
>
> On Mon, May 11, 2020 at 10:00 AM Ganesh Raju <ga...@linaro.org>
> wrote:
>
> Yes that's what Bigtop did. BTW, Apache Bigtop already has hadoop and 30
> other components.
>
> Ganesh
>
> On Mon, May 11, 2020 at 9:56 AM Matt Andruff <ma...@gmail.com>
> wrote:
>
> I kept digging and I found that ambari supports stacks (that we have all
> seen before). You can write your own custom stack
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAMBARI%2FOverview%23Overview-Structure&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=BKlsv87RZhCVj83Y7WAUw0Dydo9jiqRIsYRpN0owXZk%3D&reserved=0>
> meaning you could write the "glue" what apache projects hortonworks had
> written to roll out a set of services. I guess this is how they got around
> being open source. I guess I'll start looking into making my own stack
> from scratch. If anyone already has one or know of one, please let me know.
>
> On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <ga...@linaro.org>
> wrote:
>
> Apache Bigtop now has mpack and works with Ambari. You could try that.
>
> Thanks,
> Ganesh
>
> On Mon, May 11, 2020 at 9:15 AM Matt Andruff <ma...@gmail.com>
> wrote:
>
> Hey I was just poking around today trying to figure out if ambari still
> works to install a cluster.
>
> I recently was able to install a Cloudera Manager installation.
>
> I was about to start an Ambari installation but when I poke the repos they
> all seem to need a user/password.
>
> Is there a non-hortonworks(cloudera) hadoop that can be still be installed
> with Amabari?
>
> If there is documentation somewhere just let me know, I couldn't find the
> answer googling as Hortonworks(cloudera) docs are all that come up.
>
> --
> Live life
> Laugh Often
>
>
>
> --
> IRC: ganeshraju@#linaro on irc.freenode.ne
> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=78S3EpIVAqTe6BbaRCxjYMtghzttYoie%2FLHwYbFYCbc%3D&reserved=0>
> t
>
>
>
> --
> Live life
> Laugh Often
>
>
>
> --
> IRC: ganeshraju@#linaro on irc.freenode.ne
> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=78S3EpIVAqTe6BbaRCxjYMtghzttYoie%2FLHwYbFYCbc%3D&reserved=0>
> t
>
>
>
> --
> IRC: ganeshraju@#linaro on irc.freenode.ne
> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=78S3EpIVAqTe6BbaRCxjYMtghzttYoie%2FLHwYbFYCbc%3D&reserved=0>
> t
>
Re: life after Hortonworks
Posted by Aaron Bossert <aa...@punchcyber.com>.
For what it is worth, I have written blueprints before, but never stacks. The documentation and tutorials for ambari stacks and blueprints are horribly out of date, incomplete, or flat out missing. Perhaps that could be an initial task for the community to undertake so that those of us who were using the Hortonworks suite of tools and were comfortable with Ambari can sever the cord, as it were....relying on commercial companies to support open source tools once their objectives have changed is rarely a good thing.
Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: Ganesh Raju <ga...@linaro.org>
Sent: Monday, May 11, 2020 12:57:50 PM
To: user@ambari.apache.org <us...@ambari.apache.org>
Subject: Re: life after Hortonworks
Here is more Apache Bigtop info
releases<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FReleases&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=QqZETOUT42xKoDSVwdOFSgIS9cyS%2BeEBH12hM7e5gdI%3D&reserved=0>
mailinglist<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbigtop.apache.org%2Fmail-lists.html&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=EyqeVHJozqXHtFh76b7KQtJ4Tl%2FKhsHaVN6EBiW3ev0%3D&reserved=0>
list of components<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBIGTOP%2FBigtop%2B1.4.0%2BRelease&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=WmuQu3t9PBrITRGZnFCAm7i8dB0sxuqF%2BWHkOql75xc%3D&reserved=0>
Thanks
Ganesh Raju
On Mon, May 11, 2020 at 10:00 AM Ganesh Raju <ga...@linaro.org>> wrote:
Yes that's what Bigtop did. BTW, Apache Bigtop already has hadoop and 30 other components.
Ganesh
On Mon, May 11, 2020 at 9:56 AM Matt Andruff <ma...@gmail.com>> wrote:
I kept digging and I found that ambari supports stacks (that we have all seen before). You can write your own custom stack<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FAMBARI%2FOverview%23Overview-Structure&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=BKlsv87RZhCVj83Y7WAUw0Dydo9jiqRIsYRpN0owXZk%3D&reserved=0> meaning you could write the "glue" what apache projects hortonworks had written to roll out a set of services. I guess this is how they got around being open source. I guess I'll start looking into making my own stack from scratch. If anyone already has one or know of one, please let me know.
On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <ga...@linaro.org>> wrote:
Apache Bigtop now has mpack and works with Ambari. You could try that.
Thanks,
Ganesh
On Mon, May 11, 2020 at 9:15 AM Matt Andruff <ma...@gmail.com>> wrote:
Hey I was just poking around today trying to figure out if ambari still works to install a cluster.
I recently was able to install a Cloudera Manager installation.
I was about to start an Ambari installation but when I poke the repos they all seem to need a user/password.
Is there a non-hortonworks(cloudera) hadoop that can be still be installed with Amabari?
If there is documentation somewhere just let me know, I couldn't find the answer googling as Hortonworks(cloudera) docs are all that come up.
--
Live life
Laugh Often
--
IRC: ganeshraju@#linaro on irc.freenode.ne<https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=78S3EpIVAqTe6BbaRCxjYMtghzttYoie%2FLHwYbFYCbc%3D&reserved=0>t
--
Live life
Laugh Often
--
IRC: ganeshraju@#linaro on irc.freenode.ne<https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=78S3EpIVAqTe6BbaRCxjYMtghzttYoie%2FLHwYbFYCbc%3D&reserved=0>t
--
IRC: ganeshraju@#linaro on irc.freenode.ne<https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Firc.freenode.net%2F&data=01%7C01%7Caaron%40punchcyber.com%7C7996b7a75c94416ae98308d7f5cc7b00%7Cf7ca96216ba34d52a241e0ed3ea6a78a%7C0&sdata=78S3EpIVAqTe6BbaRCxjYMtghzttYoie%2FLHwYbFYCbc%3D&reserved=0>t
Re: life after Hortonworks
Posted by Ganesh Raju <ga...@linaro.org>.
Here is more Apache Bigtop info
releases <https://cwiki.apache.org/confluence/display/BIGTOP/Releases>
mailinglist <https://bigtop.apache.org/mail-lists.html>
list of components
<https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+1.4.0+Release>
Thanks
Ganesh Raju
On Mon, May 11, 2020 at 10:00 AM Ganesh Raju <ga...@linaro.org> wrote:
> Yes that's what Bigtop did. BTW, Apache Bigtop already has hadoop and 30
> other components.
>
> Ganesh
>
> On Mon, May 11, 2020 at 9:56 AM Matt Andruff <ma...@gmail.com>
> wrote:
>
>> I kept digging and I found that ambari supports stacks (that we have all
>> seen before). You can write your own custom stack
>> <https://cwiki.apache.org/confluence/display/AMBARI/Overview#Overview-Structure>
>> meaning you could write the "glue" what apache projects hortonworks had
>> written to roll out a set of services. I guess this is how they got around
>> being open source. I guess I'll start looking into making my own stack
>> from scratch. If anyone already has one or know of one, please let me know.
>>
>> On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <ga...@linaro.org>
>> wrote:
>>
>>> Apache Bigtop now has mpack and works with Ambari. You could try that.
>>>
>>> Thanks,
>>> Ganesh
>>>
>>> On Mon, May 11, 2020 at 9:15 AM Matt Andruff <ma...@gmail.com>
>>> wrote:
>>>
>>>> Hey I was just poking around today trying to figure out if ambari still
>>>> works to install a cluster.
>>>>
>>>> I recently was able to install a Cloudera Manager installation.
>>>>
>>>> I was about to start an Ambari installation but when I poke the repos
>>>> they all seem to need a user/password.
>>>>
>>>> Is there a non-hortonworks(cloudera) hadoop that can be still be
>>>> installed with Amabari?
>>>>
>>>> If there is documentation somewhere just let me know, I couldn't find
>>>> the answer googling as Hortonworks(cloudera) docs are all that come up.
>>>>
>>>> --
>>>> Live life
>>>> Laugh Often
>>>>
>>>
>>>
>>> --
>>> IRC: ganeshraju@#linaro on irc.freenode.ne <http://irc.freenode.net/>t
>>>
>>
>>
>> --
>> Live life
>> Laugh Often
>>
>
>
> --
> IRC: ganeshraju@#linaro on irc.freenode.ne <http://irc.freenode.net/>t
>
--
IRC: ganeshraju@#linaro on irc.freenode.ne <http://irc.freenode.net/>t
Re: life after Hortonworks
Posted by Ganesh Raju <ga...@linaro.org>.
Bigtopers
Please feel free to jump in / correct me.
Hi Matt
Agree, documentation could be improved.
https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop+1.2.0
Here is the link <https://ci.bigtop.apache.org/job/Bigtop-trunk-packages/>to
master branch CI on various distro.
Bigtop Docker hub <https://hub.docker.com/u/bigtop>
Additional documentation:
Here is a talk <https://www.youtube.com/watch?v=DudUj0McwHQ>that walks
through the build and deployment process.
https://collaborate.linaro.org/display/BDTS/How+to+%27Consume%27+a+Bigtop+release+-+to+get+a+repository+URL
https://collaborate.linaro.org/display/BDTS/Bigtop+%28v1.3.0%29+Build%2C+Smoketest%2C+and+Deploy+on+Multiple+Physical+Machines
Thanks,
Ganesh
On Tue, May 12, 2020 at 1:26 PM Matt Andruff <ma...@gmail.com> wrote:
> So, looking through this there is not a clear path from "Never worked with
> Bigtop ---> installing BigTop on Ambari"
>
> The README is a great first step, but was totally written by someone that
> already super familiar with BigTop. The Youtube video was a genius idea as
> well and a huge help. (I wished it didn't use a private repo, cause that
> steps seems to be a mystery to me still.)
>
> But I'm totally will to help make the newbie - to - big top installed, if
> you guys can just point me in the right direction.
>
> Where is the right place for the page?
>
>
> https://cwiki.apache.org/confluence/display/BIGTOP/Deployment+and+Integration+Testing
>
> or
>
> https://cwiki.apache.org/confluence/display/BIGTOP/Getting+Started
>
> or Both (under one with a link to it from the other?)
>
> Thanks
>
> Matt
>
> On Tue, May 12, 2020 at 1:16 PM Matt Andruff <ma...@gmail.com>
> wrote:
>
>> Wow thanks a lot!
>>
>> On Tue, May 12, 2020 at 12:17 PM Ganesh Raju <ga...@linaro.org>
>> wrote:
>>
>>> Matt
>>>
>>> Here is the JIRA <https://issues.apache.org/jira/browse/BIGTOP-3211>for
>>> the work done on Bigtop on Ambari mpack. Also BIGTOP-3264
>>> <https://issues.apache.org/jira/browse/BIGTOP-3264>, AMBARI-25366
>>> <https://issues.apache.org/jira/browse/AMBARI-25366>
>>> Here is a helpful youtube <https://www.youtube.com/watch?v=vnyUQtF8ZyM>video
>>> on using Ambari to install Bigtop stack. Here is a talk
>>> <https://www.youtube.com/watch?v=XhC7_IYRPxI>about it as well. And here
>>> is a wiki
>>> <https://github.com/apache/bigtop/pull/555/commits/3b3af11341806567d5edf7a7cfbe20051c074988>page
>>> on instructions.
>>> Yes please, i would appreciate it if you can ask in the Bigtop dev
>>> mailing list and you will get lots of help.
>>>
>>> Univ of Michigan is an example customer who is using Ambari mpack with
>>> Bigtop stack in production.
>>>
>>> Thanks,
>>> Ganesh
>>>
>>> On Tue, May 12, 2020 at 10:57 AM Matt Andruff <ma...@gmail.com>
>>> wrote:
>>>
>>>> thanks,
>>>>
>>>> It may be easier to work with getting big top running on Ambari. I am
>>>> sorry but I thought I went through the site pretty thoroughly. Where did
>>>> you think the big top stack was? Is there something newer than the 0.8 one
>>>> list in the ambari repo? Or should I just push this question to the dev
>>>> list?
>>>>
>>>> On Mon, May 11, 2020 at 12:51 PM Ganesh Raju <ga...@linaro.org>
>>>> wrote:
>>>>
>>>>> Stable version is v1.4. Master is at v1.5
>>>>>
>>>>> releases <https://cwiki.apache.org/confluence/display/BIGTOP/Releases>
>>>>> github <https://github.com/apache/bigtop>
>>>>> mailinglist <https://bigtop.apache.org/mail-lists.html>
>>>>> components
>>>>> <https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+1.4.0+Release>
>>>>>
>>>>> I would suggest signup for dev mailing list
>>>>> Please ask questions there and I am willing to help with all you need.
>>>>> You are welcome to contribute towards it as well. Bigtop will be happy to
>>>>> add any new component that you think would be important.
>>>>>
>>>>> Thanks,
>>>>> Ganesh
>>>>>
>>>>> On Mon, May 11, 2020 at 10:52 AM Matt Andruff <ma...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Can you share the link? I can only see them in the git repo with
>>>>>> bigtop. 0.8 and that's kinda old.
>>>>>>
>>>>>> On Mon, May 11, 2020 at 11:01 AM Ganesh Raju <ga...@linaro.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Yes that's what Bigtop did. BTW, Apache Bigtop already has hadoop
>>>>>>> and 30 other components.
>>>>>>>
>>>>>>> Ganesh
>>>>>>>
>>>>>>> On Mon, May 11, 2020 at 9:56 AM Matt Andruff <ma...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I kept digging and I found that ambari supports stacks (that we
>>>>>>>> have all seen before). You can write your own custom stack
>>>>>>>> <https://cwiki.apache.org/confluence/display/AMBARI/Overview#Overview-Structure>
>>>>>>>> meaning you could write the "glue" what apache projects hortonworks had
>>>>>>>> written to roll out a set of services. I guess this is how they got around
>>>>>>>> being open source. I guess I'll start looking into making my own stack
>>>>>>>> from scratch. If anyone already has one or know of one, please let me know.
>>>>>>>>
>>>>>>>> On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <
>>>>>>>> ganesh.raju@linaro.org> wrote:
>>>>>>>>
>>>>>>>>> Apache Bigtop now has mpack and works with Ambari. You could try
>>>>>>>>> that.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Ganesh
>>>>>>>>>
>>>>>>>>> On Mon, May 11, 2020 at 9:15 AM Matt Andruff <
>>>>>>>>> matt.andruff@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hey I was just poking around today trying to figure out if ambari
>>>>>>>>>> still works to install a cluster.
>>>>>>>>>>
>>>>>>>>>> I recently was able to install a Cloudera Manager installation.
>>>>>>>>>>
>>>>>>>>>> I was about to start an Ambari installation but when I poke the
>>>>>>>>>> repos they all seem to need a user/password.
>>>>>>>>>>
>>>>>>>>>> Is there a non-hortonworks(cloudera) hadoop that can be still be
>>>>>>>>>> installed with Amabari?
>>>>>>>>>>
>>>>>>>>>> If there is documentation somewhere just let me know, I couldn't
>>>>>>>>>> find the answer googling as Hortonworks(cloudera) docs are all that come up.
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Live life
>>>>>>>>>> Laugh Often
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> IRC: ganeshraju@#linaro on irc.freenode.ne
>>>>>>>>> <http://irc.freenode.net/>t
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Live life
>>>>>>>> Laugh Often
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> IRC: ganeshraju@#linaro on irc.freenode.ne
>>>>>>> <http://irc.freenode.net/>t
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Live life
>>>>>> Laugh Often
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> IRC: ganeshraju@#linaro on irc.freenode.ne <http://irc.freenode.net/>t
>>>>>
>>>>
>>>>
>>>> --
>>>> Live life
>>>> Laugh Often
>>>>
>>>
>>>
>>> --
>>> IRC: ganeshraju@#linaro on irc.freenode.ne <http://irc.freenode.net/>t
>>>
>>
>>
>> --
>> Live life
>> Laugh Often
>>
>
>
> --
> Live life
> Laugh Often
>
--
IRC: ganeshraju@#linaro on irc.freenode.ne <http://irc.freenode.net/>t
Re: life after Hortonworks
Posted by Ganesh Raju <ga...@linaro.org>.
Matt
Here is the JIRA <https://issues.apache.org/jira/browse/BIGTOP-3211>for the
work done on Bigtop on Ambari mpack. Also BIGTOP-3264
<https://issues.apache.org/jira/browse/BIGTOP-3264>, AMBARI-25366
<https://issues.apache.org/jira/browse/AMBARI-25366>
Here is a helpful youtube <https://www.youtube.com/watch?v=vnyUQtF8ZyM>video
on using Ambari to install Bigtop stack. Here is a talk
<https://www.youtube.com/watch?v=XhC7_IYRPxI>about it as well. And here is
a wiki
<https://github.com/apache/bigtop/pull/555/commits/3b3af11341806567d5edf7a7cfbe20051c074988>page
on instructions.
Yes please, i would appreciate it if you can ask in the Bigtop dev mailing
list and you will get lots of help.
Univ of Michigan is an example customer who is using Ambari mpack with
Bigtop stack in production.
Thanks,
Ganesh
On Tue, May 12, 2020 at 10:57 AM Matt Andruff <ma...@gmail.com>
wrote:
> thanks,
>
> It may be easier to work with getting big top running on Ambari. I am
> sorry but I thought I went through the site pretty thoroughly. Where did
> you think the big top stack was? Is there something newer than the 0.8 one
> list in the ambari repo? Or should I just push this question to the dev
> list?
>
> On Mon, May 11, 2020 at 12:51 PM Ganesh Raju <ga...@linaro.org>
> wrote:
>
>> Stable version is v1.4. Master is at v1.5
>>
>> releases <https://cwiki.apache.org/confluence/display/BIGTOP/Releases>
>> github <https://github.com/apache/bigtop>
>> mailinglist <https://bigtop.apache.org/mail-lists.html>
>> components
>> <https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+1.4.0+Release>
>>
>> I would suggest signup for dev mailing list
>> Please ask questions there and I am willing to help with all you need.
>> You are welcome to contribute towards it as well. Bigtop will be happy to
>> add any new component that you think would be important.
>>
>> Thanks,
>> Ganesh
>>
>> On Mon, May 11, 2020 at 10:52 AM Matt Andruff <ma...@gmail.com>
>> wrote:
>>
>>> Can you share the link? I can only see them in the git repo with
>>> bigtop. 0.8 and that's kinda old.
>>>
>>> On Mon, May 11, 2020 at 11:01 AM Ganesh Raju <ga...@linaro.org>
>>> wrote:
>>>
>>>> Yes that's what Bigtop did. BTW, Apache Bigtop already has hadoop and
>>>> 30 other components.
>>>>
>>>> Ganesh
>>>>
>>>> On Mon, May 11, 2020 at 9:56 AM Matt Andruff <ma...@gmail.com>
>>>> wrote:
>>>>
>>>>> I kept digging and I found that ambari supports stacks (that we have
>>>>> all seen before). You can write your own custom stack
>>>>> <https://cwiki.apache.org/confluence/display/AMBARI/Overview#Overview-Structure>
>>>>> meaning you could write the "glue" what apache projects hortonworks had
>>>>> written to roll out a set of services. I guess this is how they got around
>>>>> being open source. I guess I'll start looking into making my own stack
>>>>> from scratch. If anyone already has one or know of one, please let me know.
>>>>>
>>>>> On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <ga...@linaro.org>
>>>>> wrote:
>>>>>
>>>>>> Apache Bigtop now has mpack and works with Ambari. You could try
>>>>>> that.
>>>>>>
>>>>>> Thanks,
>>>>>> Ganesh
>>>>>>
>>>>>> On Mon, May 11, 2020 at 9:15 AM Matt Andruff <ma...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hey I was just poking around today trying to figure out if ambari
>>>>>>> still works to install a cluster.
>>>>>>>
>>>>>>> I recently was able to install a Cloudera Manager installation.
>>>>>>>
>>>>>>> I was about to start an Ambari installation but when I poke the
>>>>>>> repos they all seem to need a user/password.
>>>>>>>
>>>>>>> Is there a non-hortonworks(cloudera) hadoop that can be still be
>>>>>>> installed with Amabari?
>>>>>>>
>>>>>>> If there is documentation somewhere just let me know, I couldn't
>>>>>>> find the answer googling as Hortonworks(cloudera) docs are all that come up.
>>>>>>>
>>>>>>> --
>>>>>>> Live life
>>>>>>> Laugh Often
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> IRC: ganeshraju@#linaro on irc.freenode.ne <http://irc.freenode.net/>
>>>>>> t
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Live life
>>>>> Laugh Often
>>>>>
>>>>
>>>>
>>>> --
>>>> IRC: ganeshraju@#linaro on irc.freenode.ne <http://irc.freenode.net/>t
>>>>
>>>
>>>
>>> --
>>> Live life
>>> Laugh Often
>>>
>>
>>
>> --
>> IRC: ganeshraju@#linaro on irc.freenode.ne <http://irc.freenode.net/>t
>>
>
>
> --
> Live life
> Laugh Often
>
--
IRC: ganeshraju@#linaro on irc.freenode.ne <http://irc.freenode.net/>t
Re: life after Hortonworks
Posted by Ganesh Raju <ga...@linaro.org>.
Yes that's what Bigtop did. BTW, Apache Bigtop already has hadoop and 30
other components.
Ganesh
On Mon, May 11, 2020 at 9:56 AM Matt Andruff <ma...@gmail.com> wrote:
> I kept digging and I found that ambari supports stacks (that we have all
> seen before). You can write your own custom stack
> <https://cwiki.apache.org/confluence/display/AMBARI/Overview#Overview-Structure>
> meaning you could write the "glue" what apache projects hortonworks had
> written to roll out a set of services. I guess this is how they got around
> being open source. I guess I'll start looking into making my own stack
> from scratch. If anyone already has one or know of one, please let me know.
>
> On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <ga...@linaro.org>
> wrote:
>
>> Apache Bigtop now has mpack and works with Ambari. You could try that.
>>
>> Thanks,
>> Ganesh
>>
>> On Mon, May 11, 2020 at 9:15 AM Matt Andruff <ma...@gmail.com>
>> wrote:
>>
>>> Hey I was just poking around today trying to figure out if ambari still
>>> works to install a cluster.
>>>
>>> I recently was able to install a Cloudera Manager installation.
>>>
>>> I was about to start an Ambari installation but when I poke the repos
>>> they all seem to need a user/password.
>>>
>>> Is there a non-hortonworks(cloudera) hadoop that can be still be
>>> installed with Amabari?
>>>
>>> If there is documentation somewhere just let me know, I couldn't find
>>> the answer googling as Hortonworks(cloudera) docs are all that come up.
>>>
>>> --
>>> Live life
>>> Laugh Often
>>>
>>
>>
>> --
>> IRC: ganeshraju@#linaro on irc.freenode.ne <http://irc.freenode.net/>t
>>
>
>
> --
> Live life
> Laugh Often
>
--
IRC: ganeshraju@#linaro on irc.freenode.ne <http://irc.freenode.net/>t
Re: life after Hortonworks
Posted by Matt Andruff <ma...@gmail.com>.
I kept digging and I found that ambari supports stacks (that we have all
seen before). You can write your own custom stack
<https://cwiki.apache.org/confluence/display/AMBARI/Overview#Overview-Structure>
meaning you could write the "glue" what apache projects hortonworks had
written to roll out a set of services. I guess this is how they got around
being open source. I guess I'll start looking into making my own stack
from scratch. If anyone already has one or know of one, please let me know.
On Mon, May 11, 2020 at 10:41 AM Ganesh Raju <ga...@linaro.org> wrote:
> Apache Bigtop now has mpack and works with Ambari. You could try that.
>
> Thanks,
> Ganesh
>
> On Mon, May 11, 2020 at 9:15 AM Matt Andruff <ma...@gmail.com>
> wrote:
>
>> Hey I was just poking around today trying to figure out if ambari still
>> works to install a cluster.
>>
>> I recently was able to install a Cloudera Manager installation.
>>
>> I was about to start an Ambari installation but when I poke the repos
>> they all seem to need a user/password.
>>
>> Is there a non-hortonworks(cloudera) hadoop that can be still be
>> installed with Amabari?
>>
>> If there is documentation somewhere just let me know, I couldn't find the
>> answer googling as Hortonworks(cloudera) docs are all that come up.
>>
>> --
>> Live life
>> Laugh Often
>>
>
>
> --
> IRC: ganeshraju@#linaro on irc.freenode.ne <http://irc.freenode.net/>t
>
--
Live life
Laugh Often
Re: life after Hortonworks
Posted by Ganesh Raju <ga...@linaro.org>.
Apache Bigtop now has mpack and works with Ambari. You could try that.
Thanks,
Ganesh
On Mon, May 11, 2020 at 9:15 AM Matt Andruff <ma...@gmail.com> wrote:
> Hey I was just poking around today trying to figure out if ambari still
> works to install a cluster.
>
> I recently was able to install a Cloudera Manager installation.
>
> I was about to start an Ambari installation but when I poke the repos they
> all seem to need a user/password.
>
> Is there a non-hortonworks(cloudera) hadoop that can be still be installed
> with Amabari?
>
> If there is documentation somewhere just let me know, I couldn't find the
> answer googling as Hortonworks(cloudera) docs are all that come up.
>
> --
> Live life
> Laugh Often
>
--
IRC: ganeshraju@#linaro on irc.freenode.ne <http://irc.freenode.net/>t