You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by Andres Danter <ad...@gmail.com> on 2013/04/25 08:19:38 UTC
GSoC proposal: Ambari for Accumulo - forever : )
Hi Billie,
Here is a draft of my proposal. I've yet to write the technical approach
section (have not idea what I'm going to put there) and the "about me"
section. I definitely appreciate any feedback from you and anyone in the
development group.
Thank you,
Andres
Re: GSoC proposal: Ambari for Accumulo - forever : )
Posted by Keith Turner <ke...@deenlo.com>.
Andres,
I like your proposal.
3.1.1 loggers do not exists in 1.5 ... so would only need to consider them
if working w/ 1.4 . I suppose you would want to work Accumulo 1.4, since
that the current stable release.
3.1.3
Accumulo supports reading system config from xml local file and then
zookeeper (whats set in zookeeper takes precedence). The nice thing about
setting something a config zookeeper, is that process restart is not
required.
For section 3.2, are you proposing adding features to the existing Ambari
GUI? If so, are the Accumulo specific?
3.3 I know very little about Ambari. But I have read that it does have a
REST API. Can this be used from the API currently?
It seems that Ambari relies on RPMs. Its possible you may have to spend
time creating Accumulo RPMs that are useful for Ambari. You may want to
think about allocating some time for this. Christopher and John can give
you more info about the current state of Accumulo RPMs.
Keith
On Thu, Apr 25, 2013 at 10:09 AM, Andres Danter <ad...@gmail.com> wrote:
> Yup. Looks like that's the way to go. Here is the link to the document.
> Let me know if you have any problems accessing it.
>
>
> https://docs.google.com/file/d/0BxBY29P7A1RgVnZscGcycExDbEU/edit?usp=sharing
>
>
> On Thu, Apr 25, 2013 at 9:51 AM, Keith Turner <ke...@deenlo.com> wrote:
>
> > Nothing. I suppose the mailing list is stripping it off. Could post it
> as
> > an attachment to the ticket or put it on google drive or dropbox and
> send a
> > link.
> >
> >
> > On Thu, Apr 25, 2013 at 9:44 AM, Andres Danter <ad...@gmail.com>
> wrote:
> >
> > > Weird. I see the attachment on the email I sent, which means that it
> > > might have been stripped off. Here it is again. Let me know if you
> > still
> > > don't see it.
> > >
> > > Thanks,
> > >
> > > Andres
> > >
> > >
> > > On Thu, Apr 25, 2013 at 9:18 AM, Keith Turner <ke...@deenlo.com>
> wrote:
> > >
> > >> was there supposed to be an attachment?
> > >>
> > >>
> > >> On Thu, Apr 25, 2013 at 2:19 AM, Andres Danter <ad...@gmail.com>
> > wrote:
> > >>
> > >> > Hi Billie,
> > >> >
> > >> > Here is a draft of my proposal. I've yet to write the technical
> > >> approach
> > >> > section (have not idea what I'm going to put there) and the "about
> me"
> > >> > section. I definitely appreciate any feedback from you and anyone
> in
> > >> the
> > >> > development group.
> > >> >
> > >> > Thank you,
> > >> >
> > >> > Andres
> > >> >
> > >>
> > >
> > >
> >
>
Re: GSoC proposal: Ambari for Accumulo - forever : )
Posted by Andres Danter <ad...@gmail.com>.
Will do, Billie. Thanks.
Josh, thanks for the clarification on the loggers and 1.5.
On Thu, Apr 25, 2013 at 12:40 PM, Billie Rinaldi
<bi...@gmail.com>wrote:
> Andres, the proposal looks good, as do Keith's comments. I would suggest
> also sending it to the Ambari dev list to see if they have any additional
> comments.
>
> Billie
>
>
> On Thu, Apr 25, 2013 at 7:09 AM, Andres Danter <ad...@gmail.com> wrote:
>
> > Yup. Looks like that's the way to go. Here is the link to the document.
> > Let me know if you have any problems accessing it.
> >
> >
> >
> https://docs.google.com/file/d/0BxBY29P7A1RgVnZscGcycExDbEU/edit?usp=sharing
> >
> >
> > On Thu, Apr 25, 2013 at 9:51 AM, Keith Turner <ke...@deenlo.com> wrote:
> >
> > > Nothing. I suppose the mailing list is stripping it off. Could post
> it
> > as
> > > an attachment to the ticket or put it on google drive or dropbox and
> > send a
> > > link.
> > >
> > >
> > > On Thu, Apr 25, 2013 at 9:44 AM, Andres Danter <ad...@gmail.com>
> > wrote:
> > >
> > > > Weird. I see the attachment on the email I sent, which means that it
> > > > might have been stripped off. Here it is again. Let me know if you
> > > still
> > > > don't see it.
> > > >
> > > > Thanks,
> > > >
> > > > Andres
> > > >
> > > >
> > > > On Thu, Apr 25, 2013 at 9:18 AM, Keith Turner <ke...@deenlo.com>
> > wrote:
> > > >
> > > >> was there supposed to be an attachment?
> > > >>
> > > >>
> > > >> On Thu, Apr 25, 2013 at 2:19 AM, Andres Danter <ad...@gmail.com>
> > > wrote:
> > > >>
> > > >> > Hi Billie,
> > > >> >
> > > >> > Here is a draft of my proposal. I've yet to write the technical
> > > >> approach
> > > >> > section (have not idea what I'm going to put there) and the "about
> > me"
> > > >> > section. I definitely appreciate any feedback from you and anyone
> > in
> > > >> the
> > > >> > development group.
> > > >> >
> > > >> > Thank you,
> > > >> >
> > > >> > Andres
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>
Re: GSoC proposal: Ambari for Accumulo - forever : )
Posted by Billie Rinaldi <bi...@gmail.com>.
Andres, the proposal looks good, as do Keith's comments. I would suggest
also sending it to the Ambari dev list to see if they have any additional
comments.
Billie
On Thu, Apr 25, 2013 at 7:09 AM, Andres Danter <ad...@gmail.com> wrote:
> Yup. Looks like that's the way to go. Here is the link to the document.
> Let me know if you have any problems accessing it.
>
>
> https://docs.google.com/file/d/0BxBY29P7A1RgVnZscGcycExDbEU/edit?usp=sharing
>
>
> On Thu, Apr 25, 2013 at 9:51 AM, Keith Turner <ke...@deenlo.com> wrote:
>
> > Nothing. I suppose the mailing list is stripping it off. Could post it
> as
> > an attachment to the ticket or put it on google drive or dropbox and
> send a
> > link.
> >
> >
> > On Thu, Apr 25, 2013 at 9:44 AM, Andres Danter <ad...@gmail.com>
> wrote:
> >
> > > Weird. I see the attachment on the email I sent, which means that it
> > > might have been stripped off. Here it is again. Let me know if you
> > still
> > > don't see it.
> > >
> > > Thanks,
> > >
> > > Andres
> > >
> > >
> > > On Thu, Apr 25, 2013 at 9:18 AM, Keith Turner <ke...@deenlo.com>
> wrote:
> > >
> > >> was there supposed to be an attachment?
> > >>
> > >>
> > >> On Thu, Apr 25, 2013 at 2:19 AM, Andres Danter <ad...@gmail.com>
> > wrote:
> > >>
> > >> > Hi Billie,
> > >> >
> > >> > Here is a draft of my proposal. I've yet to write the technical
> > >> approach
> > >> > section (have not idea what I'm going to put there) and the "about
> me"
> > >> > section. I definitely appreciate any feedback from you and anyone
> in
> > >> the
> > >> > development group.
> > >> >
> > >> > Thank you,
> > >> >
> > >> > Andres
> > >> >
> > >>
> > >
> > >
> >
>
RE: GSoC proposal: Ambari for Accumulo - forever : )
Posted by Drew Pierce <dr...@live.com>.
would love to sink my teeth into this one and assist. code or doc u name it.
-DrewCambridge MA
> Date: Thu, 25 Apr 2013 11:40:26 -0400
> Subject: Re: GSoC proposal: Ambari for Accumulo - forever : )
> From: adanter@gmail.com
> To: dev@accumulo.apache.org
>
> Excellent feedback, Keith. Thanks.
>
> I should have mentioned the REST API in the proposal. I'm not sure if I
> can use it for the CLI. I will certainly try, since I would like to adhere
> to the Ambari architecture as much as possible.
>
> As for your other points:
>
> *3.1.1 loggers do not exists in 1.5 ... so would only need to consider them
> if working w/ 1.4 . I suppose you would want to work Accumulo 1.4, since
> that the current stable release.*
>
> Yes. I was going to work with Accumulo 1.4, but it is good to know that the
> loggers are not supported in 1.5.
>
> *3.1.3
> Accumulo supports reading system config from xml local file and then
> zookeeper (whats set in zookeeper takes precedence). The nice thing about
> setting something a config zookeeper, is that process restart is not
> required.*
>
> Good to know this too. Does this only affect Zookeeper? I should
> distinguish between processes that may need restarts to load a
> configuration vs those that do not. Is the Zookeeper process the only one
> like that?
>
>
> *For section 3.2, are you proposing adding features to the existing Ambari
> GUI? If so, are the Accumulo specific?*
>
> I am proposing to add features to the existing Ambari GUI. The idea here
> is that something like user access controls can be generalized so that any
> Hadoop component being managed by Ambari is able to utilize them. I'm
> hoping that the Ambari PMC is OK with me doing this. I do not want to
> introduce any Accumulo-specific features that would make no sense for other
> components.
> *
> It seems that Ambari relies on RPMs. Its possible you may have to spend
> time creating Accumulo RPMs that are useful for Ambari. You may want to
> think about allocating some time for this. Christopher and John can give
> you more info about the current state of Accumulo RPMs.*
>
> This is a very good point. While building RPMs is not difficult, it does
> require time to setup the configuration and test the builds. I will
> include time for this in the proposal.
>
> Thanks again, Keith.
>
> Andres
>
>
>
>
> On Thu, Apr 25, 2013 at 10:54 AM, Keith Turner <ke...@deenlo.com> wrote:
>
> > About the REST API, I meant to ask if this could be used from the command
> > line currently? Or is too cumbersome?
> >
> >
> > On Thu, Apr 25, 2013 at 10:09 AM, Andres Danter <ad...@gmail.com> wrote:
> >
> > > Yup. Looks like that's the way to go. Here is the link to the document.
> > > Let me know if you have any problems accessing it.
> > >
> > >
> > >
> > https://docs.google.com/file/d/0BxBY29P7A1RgVnZscGcycExDbEU/edit?usp=sharing
> > >
> > >
> > > On Thu, Apr 25, 2013 at 9:51 AM, Keith Turner <ke...@deenlo.com> wrote:
> > >
> > > > Nothing. I suppose the mailing list is stripping it off. Could post
> > it
> > > as
> > > > an attachment to the ticket or put it on google drive or dropbox and
> > > send a
> > > > link.
> > > >
> > > >
> > > > On Thu, Apr 25, 2013 at 9:44 AM, Andres Danter <ad...@gmail.com>
> > > wrote:
> > > >
> > > > > Weird. I see the attachment on the email I sent, which means that it
> > > > > might have been stripped off. Here it is again. Let me know if you
> > > > still
> > > > > don't see it.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Andres
> > > > >
> > > > >
> > > > > On Thu, Apr 25, 2013 at 9:18 AM, Keith Turner <ke...@deenlo.com>
> > > wrote:
> > > > >
> > > > >> was there supposed to be an attachment?
> > > > >>
> > > > >>
> > > > >> On Thu, Apr 25, 2013 at 2:19 AM, Andres Danter <ad...@gmail.com>
> > > > wrote:
> > > > >>
> > > > >> > Hi Billie,
> > > > >> >
> > > > >> > Here is a draft of my proposal. I've yet to write the technical
> > > > >> approach
> > > > >> > section (have not idea what I'm going to put there) and the "about
> > > me"
> > > > >> > section. I definitely appreciate any feedback from you and anyone
> > > in
> > > > >> the
> > > > >> > development group.
> > > > >> >
> > > > >> > Thank you,
> > > > >> >
> > > > >> > Andres
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
Re: GSoC proposal: Ambari for Accumulo - forever : )
Posted by Andres Danter <ad...@gmail.com>.
Hi Drew,
Sorry for replying so late. Unfortunately, this project is a solo thing.
Hopefully we can team up on another project in the future.
Thanks for the support.
Andres
On Fri, Apr 26, 2013 at 11:01 AM, Drew Pierce <dr...@live.com> wrote:
> Not sure if these are solo initiatives or those needing assistance. Let me
> know if it is the latter :)
>
>
> > Date: Fri, 26 Apr 2013 10:47:21 -0400
> > Subject: Re: GSoC proposal: Ambari for Accumulo - forever : )
> > From: adanter@gmail.com
> > To: dev@accumulo.apache.org
> >
> > Thanks, Drew, for the enthusiam.
> >
> > And thanks, John, for the good information. It is definitely an area I
> > need to look into.
> > On Apr 26, 2013 10:31 AM, "John Vines" <vi...@apache.org> wrote:
> >
> > > The init.d scripts we provide are well tested, provided you have
> accumulo
> > > installed into /usr/lib. There is probably room for improvement to
> utilize
> > > /etc/defaults/accumulo to make it configurable. This makes them not
> > > configurable with the existing RPM packaging internal to Accumulo. And
> > > Ambari uses the init.d scripts to start/stop things.
> > >
> > > As mentioned in the other thread, I'm pretty sure Ambari is pretty
> tightly
> > > integrated with Bigtop. There is an Accumulo ticket for bigtop support
> (
> > > Accumulo-138 ). I went ahead and threw a patch on there in which the
> RPM
> > > generated has been tested (the deb is lacking though). Furthermore,
> they do
> > > not use the given Accumulo init.d scripts, so there may be room for
> > > improvement in there somewhere, but I'm not sure.
> > >
> > > Should you need to work down that path more, I can definitely provide
> more
> > > guidance.
> > >
> > >
> > > On Thu, Apr 25, 2013 at 12:18 PM, Keith Turner <ke...@deenlo.com>
> wrote:
> > >
> > > > On Thu, Apr 25, 2013 at 11:40 AM, Andres Danter <ad...@gmail.com>
> > > wrote:
> > > >
> > > > > Excellent feedback, Keith. Thanks.
> > > > >
> > > > > I should have mentioned the REST API in the proposal. I'm not
> sure if
> > > I
> > > > > can use it for the CLI. I will certainly try, since I would like
> to
> > > > adhere
> > > > > to the Ambari architecture as much as possible.
> > > > >
> > > > > As for your other points:
> > > > >
> > > > > *3.1.1 loggers do not exists in 1.5 ... so would only need to
> consider
> > > > them
> > > > > if working w/ 1.4 . I suppose you would want to work Accumulo 1.4,
> > > since
> > > > > that the current stable release.*
> > > > >
> > > > > Yes. I was going to work with Accumulo 1.4, but it is good to know
> that
> > > > the
> > > > > loggers are not supported in 1.5.
> > > > >
> > > > > *3.1.3
> > > > > Accumulo supports reading system config from xml local file and
> then
> > > > > zookeeper (whats set in zookeeper takes precedence). The nice
> thing
> > > > about
> > > > > setting something a config zookeeper, is that process restart is
> not
> > > > > required.*
> > > > >
> > > > > Good to know this too. Does this only affect Zookeeper? I should
> > > > > distinguish between processes that may need restarts to load a
> > > > > configuration vs those that do not. Is the Zookeeper process the
> only
> > > > one
> > > > > like that?
> > > > >
> > > > >
> > > > Not sure about zookeeper, I think it requires a restart. Accumulo
> can
> > > > optionally store a lot of its configuration in zookeeper. When
> Accumulo
> > > > configuration stored in zookeeper is changed, Accumulo processes may
> pick
> > > > up these config changes immediately w/o restart. Some properties do
> > > > require a restart when changed in zookeeper. This is documented in
> > > > Accumulo config documentation.
> > > >
> > > >
> > > > >
> > > > > *For section 3.2, are you proposing adding features to the existing
> > > > Ambari
> > > > > GUI? If so, are the Accumulo specific?*
> > > > >
> > > > > I am proposing to add features to the existing Ambari GUI. The
> idea
> > > here
> > > > > is that something like user access controls can be generalized so
> that
> > > > any
> > > > > Hadoop component being managed by Ambari is able to utilize them.
> I'm
> > > > > hoping that the Ambari PMC is OK with me doing this. I do not
> want to
> > > > > introduce any Accumulo-specific features that would make no sense
> for
> > > > other
> > > > > components.
> > > > > *
> > > > > It seems that Ambari relies on RPMs. Its possible you may have to
> > > spend
> > > > > time creating Accumulo RPMs that are useful for Ambari. You may
> want to
> > > > > think about allocating some time for this. Christopher and John
> can
> > > give
> > > > > you more info about the current state of Accumulo RPMs.*
> > > > >
> > > > > This is a very good point. While building RPMs is not difficult,
> it
> > > does
> > > > > require time to setup the configuration and test the builds. I
> will
> > > > > include time for this in the proposal.
> > > > >
> > > >
> > > > I suppose I am really thinking about are scripts for starting and
> > > stopping
> > > > Accumulo. For an RPM to be useful it should provide good /etc/init.d
> > > > scripts for starting and stopping Accumulo processes. I am not sure
> > > what
> > > > the status of these scripts is in the various RPMs built by Accumulo.
> > > This
> > > > may not even be a concern, I am not sure how Ambari starts and stop
> > > > services. If you have not already, its something to consider when
> > > > scheduling.
> > > >
> > > >
> > > > >
> > > > > Thanks again, Keith.
> > > > >
> > > > > Andres
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Apr 25, 2013 at 10:54 AM, Keith Turner <ke...@deenlo.com>
> > > wrote:
> > > > >
> > > > > > About the REST API, I meant to ask if this could be used from the
> > > > command
> > > > > > line currently? Or is too cumbersome?
> > > > > >
> > > > > >
> > > > > > On Thu, Apr 25, 2013 at 10:09 AM, Andres Danter <
> adanter@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > Yup. Looks like that's the way to go. Here is the link to the
> > > > > document.
> > > > > > > Let me know if you have any problems accessing it.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> https://docs.google.com/file/d/0BxBY29P7A1RgVnZscGcycExDbEU/edit?usp=sharing
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Apr 25, 2013 at 9:51 AM, Keith Turner <
> keith@deenlo.com>
> > > > > wrote:
> > > > > > >
> > > > > > > > Nothing. I suppose the mailing list is stripping it off.
> Could
> > > > post
> > > > > > it
> > > > > > > as
> > > > > > > > an attachment to the ticket or put it on google drive or
> dropbox
> > > > and
> > > > > > > send a
> > > > > > > > link.
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Apr 25, 2013 at 9:44 AM, Andres Danter <
> > > adanter@gmail.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Weird. I see the attachment on the email I sent, which
> means
> > > > that
> > > > > it
> > > > > > > > > might have been stripped off. Here it is again. Let me
> know
> > > if
> > > > > you
> > > > > > > > still
> > > > > > > > > don't see it.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > >
> > > > > > > > > Andres
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Thu, Apr 25, 2013 at 9:18 AM, Keith Turner <
> > > keith@deenlo.com>
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > >> was there supposed to be an attachment?
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> On Thu, Apr 25, 2013 at 2:19 AM, Andres Danter <
> > > > adanter@gmail.com
> > > > > >
> > > > > > > > wrote:
> > > > > > > > >>
> > > > > > > > >> > Hi Billie,
> > > > > > > > >> >
> > > > > > > > >> > Here is a draft of my proposal. I've yet to write the
> > > > technical
> > > > > > > > >> approach
> > > > > > > > >> > section (have not idea what I'm going to put there) and
> the
> > > > > "about
> > > > > > > me"
> > > > > > > > >> > section. I definitely appreciate any feedback from you
> and
> > > > > anyone
> > > > > > > in
> > > > > > > > >> the
> > > > > > > > >> > development group.
> > > > > > > > >> >
> > > > > > > > >> > Thank you,
> > > > > > > > >> >
> > > > > > > > >> > Andres
> > > > > > > > >> >
> > > > > > > > >>
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
>
>
RE: GSoC proposal: Ambari for Accumulo - forever : )
Posted by Drew Pierce <dr...@live.com>.
Not sure if these are solo initiatives or those needing assistance. Let me know if it is the latter :)
> Date: Fri, 26 Apr 2013 10:47:21 -0400
> Subject: Re: GSoC proposal: Ambari for Accumulo - forever : )
> From: adanter@gmail.com
> To: dev@accumulo.apache.org
>
> Thanks, Drew, for the enthusiam.
>
> And thanks, John, for the good information. It is definitely an area I
> need to look into.
> On Apr 26, 2013 10:31 AM, "John Vines" <vi...@apache.org> wrote:
>
> > The init.d scripts we provide are well tested, provided you have accumulo
> > installed into /usr/lib. There is probably room for improvement to utilize
> > /etc/defaults/accumulo to make it configurable. This makes them not
> > configurable with the existing RPM packaging internal to Accumulo. And
> > Ambari uses the init.d scripts to start/stop things.
> >
> > As mentioned in the other thread, I'm pretty sure Ambari is pretty tightly
> > integrated with Bigtop. There is an Accumulo ticket for bigtop support (
> > Accumulo-138 ). I went ahead and threw a patch on there in which the RPM
> > generated has been tested (the deb is lacking though). Furthermore, they do
> > not use the given Accumulo init.d scripts, so there may be room for
> > improvement in there somewhere, but I'm not sure.
> >
> > Should you need to work down that path more, I can definitely provide more
> > guidance.
> >
> >
> > On Thu, Apr 25, 2013 at 12:18 PM, Keith Turner <ke...@deenlo.com> wrote:
> >
> > > On Thu, Apr 25, 2013 at 11:40 AM, Andres Danter <ad...@gmail.com>
> > wrote:
> > >
> > > > Excellent feedback, Keith. Thanks.
> > > >
> > > > I should have mentioned the REST API in the proposal. I'm not sure if
> > I
> > > > can use it for the CLI. I will certainly try, since I would like to
> > > adhere
> > > > to the Ambari architecture as much as possible.
> > > >
> > > > As for your other points:
> > > >
> > > > *3.1.1 loggers do not exists in 1.5 ... so would only need to consider
> > > them
> > > > if working w/ 1.4 . I suppose you would want to work Accumulo 1.4,
> > since
> > > > that the current stable release.*
> > > >
> > > > Yes. I was going to work with Accumulo 1.4, but it is good to know that
> > > the
> > > > loggers are not supported in 1.5.
> > > >
> > > > *3.1.3
> > > > Accumulo supports reading system config from xml local file and then
> > > > zookeeper (whats set in zookeeper takes precedence). The nice thing
> > > about
> > > > setting something a config zookeeper, is that process restart is not
> > > > required.*
> > > >
> > > > Good to know this too. Does this only affect Zookeeper? I should
> > > > distinguish between processes that may need restarts to load a
> > > > configuration vs those that do not. Is the Zookeeper process the only
> > > one
> > > > like that?
> > > >
> > > >
> > > Not sure about zookeeper, I think it requires a restart. Accumulo can
> > > optionally store a lot of its configuration in zookeeper. When Accumulo
> > > configuration stored in zookeeper is changed, Accumulo processes may pick
> > > up these config changes immediately w/o restart. Some properties do
> > > require a restart when changed in zookeeper. This is documented in
> > > Accumulo config documentation.
> > >
> > >
> > > >
> > > > *For section 3.2, are you proposing adding features to the existing
> > > Ambari
> > > > GUI? If so, are the Accumulo specific?*
> > > >
> > > > I am proposing to add features to the existing Ambari GUI. The idea
> > here
> > > > is that something like user access controls can be generalized so that
> > > any
> > > > Hadoop component being managed by Ambari is able to utilize them. I'm
> > > > hoping that the Ambari PMC is OK with me doing this. I do not want to
> > > > introduce any Accumulo-specific features that would make no sense for
> > > other
> > > > components.
> > > > *
> > > > It seems that Ambari relies on RPMs. Its possible you may have to
> > spend
> > > > time creating Accumulo RPMs that are useful for Ambari. You may want to
> > > > think about allocating some time for this. Christopher and John can
> > give
> > > > you more info about the current state of Accumulo RPMs.*
> > > >
> > > > This is a very good point. While building RPMs is not difficult, it
> > does
> > > > require time to setup the configuration and test the builds. I will
> > > > include time for this in the proposal.
> > > >
> > >
> > > I suppose I am really thinking about are scripts for starting and
> > stopping
> > > Accumulo. For an RPM to be useful it should provide good /etc/init.d
> > > scripts for starting and stopping Accumulo processes. I am not sure
> > what
> > > the status of these scripts is in the various RPMs built by Accumulo.
> > This
> > > may not even be a concern, I am not sure how Ambari starts and stop
> > > services. If you have not already, its something to consider when
> > > scheduling.
> > >
> > >
> > > >
> > > > Thanks again, Keith.
> > > >
> > > > Andres
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, Apr 25, 2013 at 10:54 AM, Keith Turner <ke...@deenlo.com>
> > wrote:
> > > >
> > > > > About the REST API, I meant to ask if this could be used from the
> > > command
> > > > > line currently? Or is too cumbersome?
> > > > >
> > > > >
> > > > > On Thu, Apr 25, 2013 at 10:09 AM, Andres Danter <ad...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > Yup. Looks like that's the way to go. Here is the link to the
> > > > document.
> > > > > > Let me know if you have any problems accessing it.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://docs.google.com/file/d/0BxBY29P7A1RgVnZscGcycExDbEU/edit?usp=sharing
> > > > > >
> > > > > >
> > > > > > On Thu, Apr 25, 2013 at 9:51 AM, Keith Turner <ke...@deenlo.com>
> > > > wrote:
> > > > > >
> > > > > > > Nothing. I suppose the mailing list is stripping it off. Could
> > > post
> > > > > it
> > > > > > as
> > > > > > > an attachment to the ticket or put it on google drive or dropbox
> > > and
> > > > > > send a
> > > > > > > link.
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Apr 25, 2013 at 9:44 AM, Andres Danter <
> > adanter@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > > Weird. I see the attachment on the email I sent, which means
> > > that
> > > > it
> > > > > > > > might have been stripped off. Here it is again. Let me know
> > if
> > > > you
> > > > > > > still
> > > > > > > > don't see it.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Andres
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Apr 25, 2013 at 9:18 AM, Keith Turner <
> > keith@deenlo.com>
> > > > > > wrote:
> > > > > > > >
> > > > > > > >> was there supposed to be an attachment?
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Thu, Apr 25, 2013 at 2:19 AM, Andres Danter <
> > > adanter@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > > >>
> > > > > > > >> > Hi Billie,
> > > > > > > >> >
> > > > > > > >> > Here is a draft of my proposal. I've yet to write the
> > > technical
> > > > > > > >> approach
> > > > > > > >> > section (have not idea what I'm going to put there) and the
> > > > "about
> > > > > > me"
> > > > > > > >> > section. I definitely appreciate any feedback from you and
> > > > anyone
> > > > > > in
> > > > > > > >> the
> > > > > > > >> > development group.
> > > > > > > >> >
> > > > > > > >> > Thank you,
> > > > > > > >> >
> > > > > > > >> > Andres
> > > > > > > >> >
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
Re: GSoC proposal: Ambari for Accumulo - forever : )
Posted by Andres Danter <ad...@gmail.com>.
Thanks, Drew, for the enthusiam.
And thanks, John, for the good information. It is definitely an area I
need to look into.
On Apr 26, 2013 10:31 AM, "John Vines" <vi...@apache.org> wrote:
> The init.d scripts we provide are well tested, provided you have accumulo
> installed into /usr/lib. There is probably room for improvement to utilize
> /etc/defaults/accumulo to make it configurable. This makes them not
> configurable with the existing RPM packaging internal to Accumulo. And
> Ambari uses the init.d scripts to start/stop things.
>
> As mentioned in the other thread, I'm pretty sure Ambari is pretty tightly
> integrated with Bigtop. There is an Accumulo ticket for bigtop support (
> Accumulo-138 ). I went ahead and threw a patch on there in which the RPM
> generated has been tested (the deb is lacking though). Furthermore, they do
> not use the given Accumulo init.d scripts, so there may be room for
> improvement in there somewhere, but I'm not sure.
>
> Should you need to work down that path more, I can definitely provide more
> guidance.
>
>
> On Thu, Apr 25, 2013 at 12:18 PM, Keith Turner <ke...@deenlo.com> wrote:
>
> > On Thu, Apr 25, 2013 at 11:40 AM, Andres Danter <ad...@gmail.com>
> wrote:
> >
> > > Excellent feedback, Keith. Thanks.
> > >
> > > I should have mentioned the REST API in the proposal. I'm not sure if
> I
> > > can use it for the CLI. I will certainly try, since I would like to
> > adhere
> > > to the Ambari architecture as much as possible.
> > >
> > > As for your other points:
> > >
> > > *3.1.1 loggers do not exists in 1.5 ... so would only need to consider
> > them
> > > if working w/ 1.4 . I suppose you would want to work Accumulo 1.4,
> since
> > > that the current stable release.*
> > >
> > > Yes. I was going to work with Accumulo 1.4, but it is good to know that
> > the
> > > loggers are not supported in 1.5.
> > >
> > > *3.1.3
> > > Accumulo supports reading system config from xml local file and then
> > > zookeeper (whats set in zookeeper takes precedence). The nice thing
> > about
> > > setting something a config zookeeper, is that process restart is not
> > > required.*
> > >
> > > Good to know this too. Does this only affect Zookeeper? I should
> > > distinguish between processes that may need restarts to load a
> > > configuration vs those that do not. Is the Zookeeper process the only
> > one
> > > like that?
> > >
> > >
> > Not sure about zookeeper, I think it requires a restart. Accumulo can
> > optionally store a lot of its configuration in zookeeper. When Accumulo
> > configuration stored in zookeeper is changed, Accumulo processes may pick
> > up these config changes immediately w/o restart. Some properties do
> > require a restart when changed in zookeeper. This is documented in
> > Accumulo config documentation.
> >
> >
> > >
> > > *For section 3.2, are you proposing adding features to the existing
> > Ambari
> > > GUI? If so, are the Accumulo specific?*
> > >
> > > I am proposing to add features to the existing Ambari GUI. The idea
> here
> > > is that something like user access controls can be generalized so that
> > any
> > > Hadoop component being managed by Ambari is able to utilize them. I'm
> > > hoping that the Ambari PMC is OK with me doing this. I do not want to
> > > introduce any Accumulo-specific features that would make no sense for
> > other
> > > components.
> > > *
> > > It seems that Ambari relies on RPMs. Its possible you may have to
> spend
> > > time creating Accumulo RPMs that are useful for Ambari. You may want to
> > > think about allocating some time for this. Christopher and John can
> give
> > > you more info about the current state of Accumulo RPMs.*
> > >
> > > This is a very good point. While building RPMs is not difficult, it
> does
> > > require time to setup the configuration and test the builds. I will
> > > include time for this in the proposal.
> > >
> >
> > I suppose I am really thinking about are scripts for starting and
> stopping
> > Accumulo. For an RPM to be useful it should provide good /etc/init.d
> > scripts for starting and stopping Accumulo processes. I am not sure
> what
> > the status of these scripts is in the various RPMs built by Accumulo.
> This
> > may not even be a concern, I am not sure how Ambari starts and stop
> > services. If you have not already, its something to consider when
> > scheduling.
> >
> >
> > >
> > > Thanks again, Keith.
> > >
> > > Andres
> > >
> > >
> > >
> > >
> > > On Thu, Apr 25, 2013 at 10:54 AM, Keith Turner <ke...@deenlo.com>
> wrote:
> > >
> > > > About the REST API, I meant to ask if this could be used from the
> > command
> > > > line currently? Or is too cumbersome?
> > > >
> > > >
> > > > On Thu, Apr 25, 2013 at 10:09 AM, Andres Danter <ad...@gmail.com>
> > > wrote:
> > > >
> > > > > Yup. Looks like that's the way to go. Here is the link to the
> > > document.
> > > > > Let me know if you have any problems accessing it.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://docs.google.com/file/d/0BxBY29P7A1RgVnZscGcycExDbEU/edit?usp=sharing
> > > > >
> > > > >
> > > > > On Thu, Apr 25, 2013 at 9:51 AM, Keith Turner <ke...@deenlo.com>
> > > wrote:
> > > > >
> > > > > > Nothing. I suppose the mailing list is stripping it off. Could
> > post
> > > > it
> > > > > as
> > > > > > an attachment to the ticket or put it on google drive or dropbox
> > and
> > > > > send a
> > > > > > link.
> > > > > >
> > > > > >
> > > > > > On Thu, Apr 25, 2013 at 9:44 AM, Andres Danter <
> adanter@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > Weird. I see the attachment on the email I sent, which means
> > that
> > > it
> > > > > > > might have been stripped off. Here it is again. Let me know
> if
> > > you
> > > > > > still
> > > > > > > don't see it.
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Andres
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Apr 25, 2013 at 9:18 AM, Keith Turner <
> keith@deenlo.com>
> > > > > wrote:
> > > > > > >
> > > > > > >> was there supposed to be an attachment?
> > > > > > >>
> > > > > > >>
> > > > > > >> On Thu, Apr 25, 2013 at 2:19 AM, Andres Danter <
> > adanter@gmail.com
> > > >
> > > > > > wrote:
> > > > > > >>
> > > > > > >> > Hi Billie,
> > > > > > >> >
> > > > > > >> > Here is a draft of my proposal. I've yet to write the
> > technical
> > > > > > >> approach
> > > > > > >> > section (have not idea what I'm going to put there) and the
> > > "about
> > > > > me"
> > > > > > >> > section. I definitely appreciate any feedback from you and
> > > anyone
> > > > > in
> > > > > > >> the
> > > > > > >> > development group.
> > > > > > >> >
> > > > > > >> > Thank you,
> > > > > > >> >
> > > > > > >> > Andres
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: GSoC proposal: Ambari for Accumulo - forever : )
Posted by John Vines <vi...@apache.org>.
The init.d scripts we provide are well tested, provided you have accumulo
installed into /usr/lib. There is probably room for improvement to utilize
/etc/defaults/accumulo to make it configurable. This makes them not
configurable with the existing RPM packaging internal to Accumulo. And
Ambari uses the init.d scripts to start/stop things.
As mentioned in the other thread, I'm pretty sure Ambari is pretty tightly
integrated with Bigtop. There is an Accumulo ticket for bigtop support (
Accumulo-138 ). I went ahead and threw a patch on there in which the RPM
generated has been tested (the deb is lacking though). Furthermore, they do
not use the given Accumulo init.d scripts, so there may be room for
improvement in there somewhere, but I'm not sure.
Should you need to work down that path more, I can definitely provide more
guidance.
On Thu, Apr 25, 2013 at 12:18 PM, Keith Turner <ke...@deenlo.com> wrote:
> On Thu, Apr 25, 2013 at 11:40 AM, Andres Danter <ad...@gmail.com> wrote:
>
> > Excellent feedback, Keith. Thanks.
> >
> > I should have mentioned the REST API in the proposal. I'm not sure if I
> > can use it for the CLI. I will certainly try, since I would like to
> adhere
> > to the Ambari architecture as much as possible.
> >
> > As for your other points:
> >
> > *3.1.1 loggers do not exists in 1.5 ... so would only need to consider
> them
> > if working w/ 1.4 . I suppose you would want to work Accumulo 1.4, since
> > that the current stable release.*
> >
> > Yes. I was going to work with Accumulo 1.4, but it is good to know that
> the
> > loggers are not supported in 1.5.
> >
> > *3.1.3
> > Accumulo supports reading system config from xml local file and then
> > zookeeper (whats set in zookeeper takes precedence). The nice thing
> about
> > setting something a config zookeeper, is that process restart is not
> > required.*
> >
> > Good to know this too. Does this only affect Zookeeper? I should
> > distinguish between processes that may need restarts to load a
> > configuration vs those that do not. Is the Zookeeper process the only
> one
> > like that?
> >
> >
> Not sure about zookeeper, I think it requires a restart. Accumulo can
> optionally store a lot of its configuration in zookeeper. When Accumulo
> configuration stored in zookeeper is changed, Accumulo processes may pick
> up these config changes immediately w/o restart. Some properties do
> require a restart when changed in zookeeper. This is documented in
> Accumulo config documentation.
>
>
> >
> > *For section 3.2, are you proposing adding features to the existing
> Ambari
> > GUI? If so, are the Accumulo specific?*
> >
> > I am proposing to add features to the existing Ambari GUI. The idea here
> > is that something like user access controls can be generalized so that
> any
> > Hadoop component being managed by Ambari is able to utilize them. I'm
> > hoping that the Ambari PMC is OK with me doing this. I do not want to
> > introduce any Accumulo-specific features that would make no sense for
> other
> > components.
> > *
> > It seems that Ambari relies on RPMs. Its possible you may have to spend
> > time creating Accumulo RPMs that are useful for Ambari. You may want to
> > think about allocating some time for this. Christopher and John can give
> > you more info about the current state of Accumulo RPMs.*
> >
> > This is a very good point. While building RPMs is not difficult, it does
> > require time to setup the configuration and test the builds. I will
> > include time for this in the proposal.
> >
>
> I suppose I am really thinking about are scripts for starting and stopping
> Accumulo. For an RPM to be useful it should provide good /etc/init.d
> scripts for starting and stopping Accumulo processes. I am not sure what
> the status of these scripts is in the various RPMs built by Accumulo. This
> may not even be a concern, I am not sure how Ambari starts and stop
> services. If you have not already, its something to consider when
> scheduling.
>
>
> >
> > Thanks again, Keith.
> >
> > Andres
> >
> >
> >
> >
> > On Thu, Apr 25, 2013 at 10:54 AM, Keith Turner <ke...@deenlo.com> wrote:
> >
> > > About the REST API, I meant to ask if this could be used from the
> command
> > > line currently? Or is too cumbersome?
> > >
> > >
> > > On Thu, Apr 25, 2013 at 10:09 AM, Andres Danter <ad...@gmail.com>
> > wrote:
> > >
> > > > Yup. Looks like that's the way to go. Here is the link to the
> > document.
> > > > Let me know if you have any problems accessing it.
> > > >
> > > >
> > > >
> > >
> >
> https://docs.google.com/file/d/0BxBY29P7A1RgVnZscGcycExDbEU/edit?usp=sharing
> > > >
> > > >
> > > > On Thu, Apr 25, 2013 at 9:51 AM, Keith Turner <ke...@deenlo.com>
> > wrote:
> > > >
> > > > > Nothing. I suppose the mailing list is stripping it off. Could
> post
> > > it
> > > > as
> > > > > an attachment to the ticket or put it on google drive or dropbox
> and
> > > > send a
> > > > > link.
> > > > >
> > > > >
> > > > > On Thu, Apr 25, 2013 at 9:44 AM, Andres Danter <ad...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > Weird. I see the attachment on the email I sent, which means
> that
> > it
> > > > > > might have been stripped off. Here it is again. Let me know if
> > you
> > > > > still
> > > > > > don't see it.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Andres
> > > > > >
> > > > > >
> > > > > > On Thu, Apr 25, 2013 at 9:18 AM, Keith Turner <ke...@deenlo.com>
> > > > wrote:
> > > > > >
> > > > > >> was there supposed to be an attachment?
> > > > > >>
> > > > > >>
> > > > > >> On Thu, Apr 25, 2013 at 2:19 AM, Andres Danter <
> adanter@gmail.com
> > >
> > > > > wrote:
> > > > > >>
> > > > > >> > Hi Billie,
> > > > > >> >
> > > > > >> > Here is a draft of my proposal. I've yet to write the
> technical
> > > > > >> approach
> > > > > >> > section (have not idea what I'm going to put there) and the
> > "about
> > > > me"
> > > > > >> > section. I definitely appreciate any feedback from you and
> > anyone
> > > > in
> > > > > >> the
> > > > > >> > development group.
> > > > > >> >
> > > > > >> > Thank you,
> > > > > >> >
> > > > > >> > Andres
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: GSoC proposal: Ambari for Accumulo - forever : )
Posted by Keith Turner <ke...@deenlo.com>.
On Thu, Apr 25, 2013 at 11:40 AM, Andres Danter <ad...@gmail.com> wrote:
> Excellent feedback, Keith. Thanks.
>
> I should have mentioned the REST API in the proposal. I'm not sure if I
> can use it for the CLI. I will certainly try, since I would like to adhere
> to the Ambari architecture as much as possible.
>
> As for your other points:
>
> *3.1.1 loggers do not exists in 1.5 ... so would only need to consider them
> if working w/ 1.4 . I suppose you would want to work Accumulo 1.4, since
> that the current stable release.*
>
> Yes. I was going to work with Accumulo 1.4, but it is good to know that the
> loggers are not supported in 1.5.
>
> *3.1.3
> Accumulo supports reading system config from xml local file and then
> zookeeper (whats set in zookeeper takes precedence). The nice thing about
> setting something a config zookeeper, is that process restart is not
> required.*
>
> Good to know this too. Does this only affect Zookeeper? I should
> distinguish between processes that may need restarts to load a
> configuration vs those that do not. Is the Zookeeper process the only one
> like that?
>
>
Not sure about zookeeper, I think it requires a restart. Accumulo can
optionally store a lot of its configuration in zookeeper. When Accumulo
configuration stored in zookeeper is changed, Accumulo processes may pick
up these config changes immediately w/o restart. Some properties do
require a restart when changed in zookeeper. This is documented in
Accumulo config documentation.
>
> *For section 3.2, are you proposing adding features to the existing Ambari
> GUI? If so, are the Accumulo specific?*
>
> I am proposing to add features to the existing Ambari GUI. The idea here
> is that something like user access controls can be generalized so that any
> Hadoop component being managed by Ambari is able to utilize them. I'm
> hoping that the Ambari PMC is OK with me doing this. I do not want to
> introduce any Accumulo-specific features that would make no sense for other
> components.
> *
> It seems that Ambari relies on RPMs. Its possible you may have to spend
> time creating Accumulo RPMs that are useful for Ambari. You may want to
> think about allocating some time for this. Christopher and John can give
> you more info about the current state of Accumulo RPMs.*
>
> This is a very good point. While building RPMs is not difficult, it does
> require time to setup the configuration and test the builds. I will
> include time for this in the proposal.
>
I suppose I am really thinking about are scripts for starting and stopping
Accumulo. For an RPM to be useful it should provide good /etc/init.d
scripts for starting and stopping Accumulo processes. I am not sure what
the status of these scripts is in the various RPMs built by Accumulo. This
may not even be a concern, I am not sure how Ambari starts and stop
services. If you have not already, its something to consider when
scheduling.
>
> Thanks again, Keith.
>
> Andres
>
>
>
>
> On Thu, Apr 25, 2013 at 10:54 AM, Keith Turner <ke...@deenlo.com> wrote:
>
> > About the REST API, I meant to ask if this could be used from the command
> > line currently? Or is too cumbersome?
> >
> >
> > On Thu, Apr 25, 2013 at 10:09 AM, Andres Danter <ad...@gmail.com>
> wrote:
> >
> > > Yup. Looks like that's the way to go. Here is the link to the
> document.
> > > Let me know if you have any problems accessing it.
> > >
> > >
> > >
> >
> https://docs.google.com/file/d/0BxBY29P7A1RgVnZscGcycExDbEU/edit?usp=sharing
> > >
> > >
> > > On Thu, Apr 25, 2013 at 9:51 AM, Keith Turner <ke...@deenlo.com>
> wrote:
> > >
> > > > Nothing. I suppose the mailing list is stripping it off. Could post
> > it
> > > as
> > > > an attachment to the ticket or put it on google drive or dropbox and
> > > send a
> > > > link.
> > > >
> > > >
> > > > On Thu, Apr 25, 2013 at 9:44 AM, Andres Danter <ad...@gmail.com>
> > > wrote:
> > > >
> > > > > Weird. I see the attachment on the email I sent, which means that
> it
> > > > > might have been stripped off. Here it is again. Let me know if
> you
> > > > still
> > > > > don't see it.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Andres
> > > > >
> > > > >
> > > > > On Thu, Apr 25, 2013 at 9:18 AM, Keith Turner <ke...@deenlo.com>
> > > wrote:
> > > > >
> > > > >> was there supposed to be an attachment?
> > > > >>
> > > > >>
> > > > >> On Thu, Apr 25, 2013 at 2:19 AM, Andres Danter <adanter@gmail.com
> >
> > > > wrote:
> > > > >>
> > > > >> > Hi Billie,
> > > > >> >
> > > > >> > Here is a draft of my proposal. I've yet to write the technical
> > > > >> approach
> > > > >> > section (have not idea what I'm going to put there) and the
> "about
> > > me"
> > > > >> > section. I definitely appreciate any feedback from you and
> anyone
> > > in
> > > > >> the
> > > > >> > development group.
> > > > >> >
> > > > >> > Thank you,
> > > > >> >
> > > > >> > Andres
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: GSoC proposal: Ambari for Accumulo - forever : )
Posted by Josh Elser <jo...@gmail.com>.
On 4/25/13 11:40 AM, Andres Danter wrote:
> Excellent feedback, Keith. Thanks.
>
> I should have mentioned the REST API in the proposal. I'm not sure if I
> can use it for the CLI. I will certainly try, since I would like to adhere
> to the Ambari architecture as much as possible.
>
> As for your other points:
>
> *3.1.1 loggers do not exists in 1.5 ... so would only need to consider them
> if working w/ 1.4 . I suppose you would want to work Accumulo 1.4, since
> that the current stable release.*
>
> Yes. I was going to work with Accumulo 1.4, but it is good to know that the
> loggers are not supported in 1.5.
Not so much "not supported" in 1.5, but more "not necessary". The
write-ahead log (mechanism to ensure data is not lost during ingest) is
backed by HDFS and managed by TServers in >=1.5 instead of a separate
Logger process (used local fs and handled replication internally) which
<=1.4 has.
>
> *3.1.3
> Accumulo supports reading system config from xml local file and then
> zookeeper (whats set in zookeeper takes precedence). The nice thing about
> setting something a config zookeeper, is that process restart is not
> required.*
>
> Good to know this too. Does this only affect Zookeeper? I should
> distinguish between processes that may need restarts to load a
> configuration vs those that do not. Is the Zookeeper process the only one
> like that?
>
>
> *For section 3.2, are you proposing adding features to the existing Ambari
> GUI? If so, are the Accumulo specific?*
>
> I am proposing to add features to the existing Ambari GUI. The idea here
> is that something like user access controls can be generalized so that any
> Hadoop component being managed by Ambari is able to utilize them. I'm
> hoping that the Ambari PMC is OK with me doing this. I do not want to
> introduce any Accumulo-specific features that would make no sense for other
> components.
> *
> It seems that Ambari relies on RPMs. Its possible you may have to spend
> time creating Accumulo RPMs that are useful for Ambari. You may want to
> think about allocating some time for this. Christopher and John can give
> you more info about the current state of Accumulo RPMs.*
>
> This is a very good point. While building RPMs is not difficult, it does
> require time to setup the configuration and test the builds. I will
> include time for this in the proposal.
>
> Thanks again, Keith.
>
> Andres
>
>
>
>
> On Thu, Apr 25, 2013 at 10:54 AM, Keith Turner <ke...@deenlo.com> wrote:
>
>> About the REST API, I meant to ask if this could be used from the command
>> line currently? Or is too cumbersome?
>>
>>
>> On Thu, Apr 25, 2013 at 10:09 AM, Andres Danter <ad...@gmail.com> wrote:
>>
>>> Yup. Looks like that's the way to go. Here is the link to the document.
>>> Let me know if you have any problems accessing it.
>>>
>>>
>>>
>> https://docs.google.com/file/d/0BxBY29P7A1RgVnZscGcycExDbEU/edit?usp=sharing
>>>
>>> On Thu, Apr 25, 2013 at 9:51 AM, Keith Turner <ke...@deenlo.com> wrote:
>>>
>>>> Nothing. I suppose the mailing list is stripping it off. Could post
>> it
>>> as
>>>> an attachment to the ticket or put it on google drive or dropbox and
>>> send a
>>>> link.
>>>>
>>>>
>>>> On Thu, Apr 25, 2013 at 9:44 AM, Andres Danter <ad...@gmail.com>
>>> wrote:
>>>>> Weird. I see the attachment on the email I sent, which means that it
>>>>> might have been stripped off. Here it is again. Let me know if you
>>>> still
>>>>> don't see it.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Andres
>>>>>
>>>>>
>>>>> On Thu, Apr 25, 2013 at 9:18 AM, Keith Turner <ke...@deenlo.com>
>>> wrote:
>>>>>> was there supposed to be an attachment?
>>>>>>
>>>>>>
>>>>>> On Thu, Apr 25, 2013 at 2:19 AM, Andres Danter <ad...@gmail.com>
>>>> wrote:
>>>>>>> Hi Billie,
>>>>>>>
>>>>>>> Here is a draft of my proposal. I've yet to write the technical
>>>>>> approach
>>>>>>> section (have not idea what I'm going to put there) and the "about
>>> me"
>>>>>>> section. I definitely appreciate any feedback from you and anyone
>>> in
>>>>>> the
>>>>>>> development group.
>>>>>>>
>>>>>>> Thank you,
>>>>>>>
>>>>>>> Andres
>>>>>>>
>>>>>
Re: GSoC proposal: Ambari for Accumulo - forever : )
Posted by Andres Danter <ad...@gmail.com>.
Excellent feedback, Keith. Thanks.
I should have mentioned the REST API in the proposal. I'm not sure if I
can use it for the CLI. I will certainly try, since I would like to adhere
to the Ambari architecture as much as possible.
As for your other points:
*3.1.1 loggers do not exists in 1.5 ... so would only need to consider them
if working w/ 1.4 . I suppose you would want to work Accumulo 1.4, since
that the current stable release.*
Yes. I was going to work with Accumulo 1.4, but it is good to know that the
loggers are not supported in 1.5.
*3.1.3
Accumulo supports reading system config from xml local file and then
zookeeper (whats set in zookeeper takes precedence). The nice thing about
setting something a config zookeeper, is that process restart is not
required.*
Good to know this too. Does this only affect Zookeeper? I should
distinguish between processes that may need restarts to load a
configuration vs those that do not. Is the Zookeeper process the only one
like that?
*For section 3.2, are you proposing adding features to the existing Ambari
GUI? If so, are the Accumulo specific?*
I am proposing to add features to the existing Ambari GUI. The idea here
is that something like user access controls can be generalized so that any
Hadoop component being managed by Ambari is able to utilize them. I'm
hoping that the Ambari PMC is OK with me doing this. I do not want to
introduce any Accumulo-specific features that would make no sense for other
components.
*
It seems that Ambari relies on RPMs. Its possible you may have to spend
time creating Accumulo RPMs that are useful for Ambari. You may want to
think about allocating some time for this. Christopher and John can give
you more info about the current state of Accumulo RPMs.*
This is a very good point. While building RPMs is not difficult, it does
require time to setup the configuration and test the builds. I will
include time for this in the proposal.
Thanks again, Keith.
Andres
On Thu, Apr 25, 2013 at 10:54 AM, Keith Turner <ke...@deenlo.com> wrote:
> About the REST API, I meant to ask if this could be used from the command
> line currently? Or is too cumbersome?
>
>
> On Thu, Apr 25, 2013 at 10:09 AM, Andres Danter <ad...@gmail.com> wrote:
>
> > Yup. Looks like that's the way to go. Here is the link to the document.
> > Let me know if you have any problems accessing it.
> >
> >
> >
> https://docs.google.com/file/d/0BxBY29P7A1RgVnZscGcycExDbEU/edit?usp=sharing
> >
> >
> > On Thu, Apr 25, 2013 at 9:51 AM, Keith Turner <ke...@deenlo.com> wrote:
> >
> > > Nothing. I suppose the mailing list is stripping it off. Could post
> it
> > as
> > > an attachment to the ticket or put it on google drive or dropbox and
> > send a
> > > link.
> > >
> > >
> > > On Thu, Apr 25, 2013 at 9:44 AM, Andres Danter <ad...@gmail.com>
> > wrote:
> > >
> > > > Weird. I see the attachment on the email I sent, which means that it
> > > > might have been stripped off. Here it is again. Let me know if you
> > > still
> > > > don't see it.
> > > >
> > > > Thanks,
> > > >
> > > > Andres
> > > >
> > > >
> > > > On Thu, Apr 25, 2013 at 9:18 AM, Keith Turner <ke...@deenlo.com>
> > wrote:
> > > >
> > > >> was there supposed to be an attachment?
> > > >>
> > > >>
> > > >> On Thu, Apr 25, 2013 at 2:19 AM, Andres Danter <ad...@gmail.com>
> > > wrote:
> > > >>
> > > >> > Hi Billie,
> > > >> >
> > > >> > Here is a draft of my proposal. I've yet to write the technical
> > > >> approach
> > > >> > section (have not idea what I'm going to put there) and the "about
> > me"
> > > >> > section. I definitely appreciate any feedback from you and anyone
> > in
> > > >> the
> > > >> > development group.
> > > >> >
> > > >> > Thank you,
> > > >> >
> > > >> > Andres
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>
Re: GSoC proposal: Ambari for Accumulo - forever : )
Posted by Keith Turner <ke...@deenlo.com>.
About the REST API, I meant to ask if this could be used from the command
line currently? Or is too cumbersome?
On Thu, Apr 25, 2013 at 10:09 AM, Andres Danter <ad...@gmail.com> wrote:
> Yup. Looks like that's the way to go. Here is the link to the document.
> Let me know if you have any problems accessing it.
>
>
> https://docs.google.com/file/d/0BxBY29P7A1RgVnZscGcycExDbEU/edit?usp=sharing
>
>
> On Thu, Apr 25, 2013 at 9:51 AM, Keith Turner <ke...@deenlo.com> wrote:
>
> > Nothing. I suppose the mailing list is stripping it off. Could post it
> as
> > an attachment to the ticket or put it on google drive or dropbox and
> send a
> > link.
> >
> >
> > On Thu, Apr 25, 2013 at 9:44 AM, Andres Danter <ad...@gmail.com>
> wrote:
> >
> > > Weird. I see the attachment on the email I sent, which means that it
> > > might have been stripped off. Here it is again. Let me know if you
> > still
> > > don't see it.
> > >
> > > Thanks,
> > >
> > > Andres
> > >
> > >
> > > On Thu, Apr 25, 2013 at 9:18 AM, Keith Turner <ke...@deenlo.com>
> wrote:
> > >
> > >> was there supposed to be an attachment?
> > >>
> > >>
> > >> On Thu, Apr 25, 2013 at 2:19 AM, Andres Danter <ad...@gmail.com>
> > wrote:
> > >>
> > >> > Hi Billie,
> > >> >
> > >> > Here is a draft of my proposal. I've yet to write the technical
> > >> approach
> > >> > section (have not idea what I'm going to put there) and the "about
> me"
> > >> > section. I definitely appreciate any feedback from you and anyone
> in
> > >> the
> > >> > development group.
> > >> >
> > >> > Thank you,
> > >> >
> > >> > Andres
> > >> >
> > >>
> > >
> > >
> >
>
Re: GSoC proposal: Ambari for Accumulo - forever : )
Posted by Andres Danter <ad...@gmail.com>.
Yup. Looks like that's the way to go. Here is the link to the document.
Let me know if you have any problems accessing it.
https://docs.google.com/file/d/0BxBY29P7A1RgVnZscGcycExDbEU/edit?usp=sharing
On Thu, Apr 25, 2013 at 9:51 AM, Keith Turner <ke...@deenlo.com> wrote:
> Nothing. I suppose the mailing list is stripping it off. Could post it as
> an attachment to the ticket or put it on google drive or dropbox and send a
> link.
>
>
> On Thu, Apr 25, 2013 at 9:44 AM, Andres Danter <ad...@gmail.com> wrote:
>
> > Weird. I see the attachment on the email I sent, which means that it
> > might have been stripped off. Here it is again. Let me know if you
> still
> > don't see it.
> >
> > Thanks,
> >
> > Andres
> >
> >
> > On Thu, Apr 25, 2013 at 9:18 AM, Keith Turner <ke...@deenlo.com> wrote:
> >
> >> was there supposed to be an attachment?
> >>
> >>
> >> On Thu, Apr 25, 2013 at 2:19 AM, Andres Danter <ad...@gmail.com>
> wrote:
> >>
> >> > Hi Billie,
> >> >
> >> > Here is a draft of my proposal. I've yet to write the technical
> >> approach
> >> > section (have not idea what I'm going to put there) and the "about me"
> >> > section. I definitely appreciate any feedback from you and anyone in
> >> the
> >> > development group.
> >> >
> >> > Thank you,
> >> >
> >> > Andres
> >> >
> >>
> >
> >
>
Re: GSoC proposal: Ambari for Accumulo - forever : )
Posted by Keith Turner <ke...@deenlo.com>.
Nothing. I suppose the mailing list is stripping it off. Could post it as
an attachment to the ticket or put it on google drive or dropbox and send a
link.
On Thu, Apr 25, 2013 at 9:44 AM, Andres Danter <ad...@gmail.com> wrote:
> Weird. I see the attachment on the email I sent, which means that it
> might have been stripped off. Here it is again. Let me know if you still
> don't see it.
>
> Thanks,
>
> Andres
>
>
> On Thu, Apr 25, 2013 at 9:18 AM, Keith Turner <ke...@deenlo.com> wrote:
>
>> was there supposed to be an attachment?
>>
>>
>> On Thu, Apr 25, 2013 at 2:19 AM, Andres Danter <ad...@gmail.com> wrote:
>>
>> > Hi Billie,
>> >
>> > Here is a draft of my proposal. I've yet to write the technical
>> approach
>> > section (have not idea what I'm going to put there) and the "about me"
>> > section. I definitely appreciate any feedback from you and anyone in
>> the
>> > development group.
>> >
>> > Thank you,
>> >
>> > Andres
>> >
>>
>
>
Re: GSoC proposal: Ambari for Accumulo - forever : )
Posted by Andres Danter <ad...@gmail.com>.
Weird. I see the attachment on the email I sent, which means that it might
have been stripped off. Here it is again. Let me know if you still don't
see it.
Thanks,
Andres
On Thu, Apr 25, 2013 at 9:18 AM, Keith Turner <ke...@deenlo.com> wrote:
> was there supposed to be an attachment?
>
>
> On Thu, Apr 25, 2013 at 2:19 AM, Andres Danter <ad...@gmail.com> wrote:
>
> > Hi Billie,
> >
> > Here is a draft of my proposal. I've yet to write the technical approach
> > section (have not idea what I'm going to put there) and the "about me"
> > section. I definitely appreciate any feedback from you and anyone in the
> > development group.
> >
> > Thank you,
> >
> > Andres
> >
>
Re: GSoC proposal: Ambari for Accumulo - forever : )
Posted by Keith Turner <ke...@deenlo.com>.
was there supposed to be an attachment?
On Thu, Apr 25, 2013 at 2:19 AM, Andres Danter <ad...@gmail.com> wrote:
> Hi Billie,
>
> Here is a draft of my proposal. I've yet to write the technical approach
> section (have not idea what I'm going to put there) and the "about me"
> section. I definitely appreciate any feedback from you and anyone in the
> development group.
>
> Thank you,
>
> Andres
>