RPMs/DEBs for Accumulo 1.6.0 via Bigtop

[Josh Elser <jo...@gmail.com>]

One rather user-facing change that we made with 1.6.0 was the removal of 
the RPM and DEB packaging from our tree. We did this because maintenance 
of that packaging code was historically a problem, and we felt like 
bigtop would be a better home for it rather then perpetuate the bit-rot.

Personally, I think we messed up by removing the code from our tree 
before bigtop has the necessary pieces in place. Since getting the code 
into bigtop is still progressing slowly[1], I've forked bigtop[2], added 
the patch and fixed some bugs that are currently in the patch.

It's fairly easy to generate the packaging if you have some time to walk 
through it. I successfully installed and ran Accumulo 1.6.0 via Bigtop 
on Centos6.5 today.

Rough outline:

1. Download the code from [2]
2. Look at the DEVNOTES [3] and install the prereqs (there may be more, 
but I tried to update the list)
3. Ensure Maven and Gradle are installed too
4. gradle hadoop-rpm
5. gradle zookeeper-rpm
6. gradle bigtop-jsvc
7. gradle bigtop-utils
8. gradle accumulo-rpm

(check out `gradle tasks` for more info on steps 4-8).

Then, install the packages which are placed in the build/ directory at 
the top level of the repo. `find` is your friend. the Accumulo RPMs and 
SRPMs both built successfully. I haven't tried building DEBs yet, so 
YMMV there. (hopefully will make some time later this week).

- Josh

[1] https://issues.apache.org/jira/browse/BIGTOP-1175
[2] https://github.com/joshelser/bigtop/tree/accumulo-support
[3] https://github.com/joshelser/bigtop/blob/accumulo-support/DEVNOTES

Re: RPMs/DEBs for Accumulo 1.6.0 via Bigtop

[Josh Elser <jo...@gmail.com>]

No worries! I don't mean to be getting in the way of Bigtop, I just know 
that I can move faster and provide the ability for people to make 
Accumulo RPM/DEBs in fewer steps, while the bigger cogs spin.

Thank you for your efforts.

WRT packagin tests, we have a number of tests already in our tree (in 
addition to lots of people who like to think about tests). I'm sure we 
can help there :)

On 6/25/14, 11:27 AM, Sean Mackrory wrote:
> Sorry it's been progressing slowly - I've been quite busy lately. I'll push
> to get that committed soon. I think the only hesitance others had to put it
> in master is there's a Bigtop release planned very soon and they want to
> start stabilizing that one - I expect it'd be merge to master as soon as
> that release is done, although packaging tests will need to be written as
> well.
>
>
> On Tue, Jun 24, 2014 at 2:06 PM, Christopher <ct...@apache.org> wrote:
>
>> While I don't agree we messed up (I think this is the correct order:
>> upstream release, then downstream packaging; and I like this distinction),
>> I'm glad you did this. Thanks for putting these together. Hopefully this
>> will benefit the BigTop users and Accumulo users interested in BigTop.
>>
>> Thanks, Josh!
>>
>>
>> --
>> Christopher L Tubbs II
>> http://gravatar.com/ctubbsii
>>
>>
>> On Tue, Jun 24, 2014 at 12:32 AM, Josh Elser <jo...@gmail.com> wrote:
>>
>>> One rather user-facing change that we made with 1.6.0 was the removal of
>>> the RPM and DEB packaging from our tree. We did this because maintenance
>> of
>>> that packaging code was historically a problem, and we felt like bigtop
>>> would be a better home for it rather then perpetuate the bit-rot.
>>>
>>> Personally, I think we messed up by removing the code from our tree
>> before
>>> bigtop has the necessary pieces in place. Since getting the code into
>>> bigtop is still progressing slowly[1], I've forked bigtop[2], added the
>>> patch and fixed some bugs that are currently in the patch.
>>>
>>> It's fairly easy to generate the packaging if you have some time to walk
>>> through it. I successfully installed and ran Accumulo 1.6.0 via Bigtop on
>>> Centos6.5 today.
>>>
>>> Rough outline:
>>>
>>> 1. Download the code from [2]
>>> 2. Look at the DEVNOTES [3] and install the prereqs (there may be more,
>>> but I tried to update the list)
>>> 3. Ensure Maven and Gradle are installed too
>>> 4. gradle hadoop-rpm
>>> 5. gradle zookeeper-rpm
>>> 6. gradle bigtop-jsvc
>>> 7. gradle bigtop-utils
>>> 8. gradle accumulo-rpm
>>>
>>> (check out `gradle tasks` for more info on steps 4-8).
>>>
>>> Then, install the packages which are placed in the build/ directory at
>> the
>>> top level of the repo. `find` is your friend. the Accumulo RPMs and SRPMs
>>> both built successfully. I haven't tried building DEBs yet, so YMMV
>> there.
>>> (hopefully will make some time later this week).
>>>
>>> - Josh
>>>
>>> [1] https://issues.apache.org/jira/browse/BIGTOP-1175
>>> [2] https://github.com/joshelser/bigtop/tree/accumulo-support
>>> [3] https://github.com/joshelser/bigtop/blob/accumulo-support/DEVNOTES
>>>
>>
>

Re: RPMs/DEBs for Accumulo 1.6.0 via Bigtop

[Sean Mackrory <ma...@gmail.com>]

Sorry it's been progressing slowly - I've been quite busy lately. I'll push
to get that committed soon. I think the only hesitance others had to put it
in master is there's a Bigtop release planned very soon and they want to
start stabilizing that one - I expect it'd be merge to master as soon as
that release is done, although packaging tests will need to be written as
well.


On Tue, Jun 24, 2014 at 2:06 PM, Christopher <ct...@apache.org> wrote:

> While I don't agree we messed up (I think this is the correct order:
> upstream release, then downstream packaging; and I like this distinction),
> I'm glad you did this. Thanks for putting these together. Hopefully this
> will benefit the BigTop users and Accumulo users interested in BigTop.
>
> Thanks, Josh!
>
>
> --
> Christopher L Tubbs II
> http://gravatar.com/ctubbsii
>
>
> On Tue, Jun 24, 2014 at 12:32 AM, Josh Elser <jo...@gmail.com> wrote:
>
> > One rather user-facing change that we made with 1.6.0 was the removal of
> > the RPM and DEB packaging from our tree. We did this because maintenance
> of
> > that packaging code was historically a problem, and we felt like bigtop
> > would be a better home for it rather then perpetuate the bit-rot.
> >
> > Personally, I think we messed up by removing the code from our tree
> before
> > bigtop has the necessary pieces in place. Since getting the code into
> > bigtop is still progressing slowly[1], I've forked bigtop[2], added the
> > patch and fixed some bugs that are currently in the patch.
> >
> > It's fairly easy to generate the packaging if you have some time to walk
> > through it. I successfully installed and ran Accumulo 1.6.0 via Bigtop on
> > Centos6.5 today.
> >
> > Rough outline:
> >
> > 1. Download the code from [2]
> > 2. Look at the DEVNOTES [3] and install the prereqs (there may be more,
> > but I tried to update the list)
> > 3. Ensure Maven and Gradle are installed too
> > 4. gradle hadoop-rpm
> > 5. gradle zookeeper-rpm
> > 6. gradle bigtop-jsvc
> > 7. gradle bigtop-utils
> > 8. gradle accumulo-rpm
> >
> > (check out `gradle tasks` for more info on steps 4-8).
> >
> > Then, install the packages which are placed in the build/ directory at
> the
> > top level of the repo. `find` is your friend. the Accumulo RPMs and SRPMs
> > both built successfully. I haven't tried building DEBs yet, so YMMV
> there.
> > (hopefully will make some time later this week).
> >
> > - Josh
> >
> > [1] https://issues.apache.org/jira/browse/BIGTOP-1175
> > [2] https://github.com/joshelser/bigtop/tree/accumulo-support
> > [3] https://github.com/joshelser/bigtop/blob/accumulo-support/DEVNOTES
> >
>

Re: RPMs/DEBs for Accumulo 1.6.0 via Bigtop

[Christopher <ct...@apache.org>]

While I don't agree we messed up (I think this is the correct order:
upstream release, then downstream packaging; and I like this distinction),
I'm glad you did this. Thanks for putting these together. Hopefully this
will benefit the BigTop users and Accumulo users interested in BigTop.

Thanks, Josh!


--
Christopher L Tubbs II
http://gravatar.com/ctubbsii


On Tue, Jun 24, 2014 at 12:32 AM, Josh Elser <jo...@gmail.com> wrote:

> One rather user-facing change that we made with 1.6.0 was the removal of
> the RPM and DEB packaging from our tree. We did this because maintenance of
> that packaging code was historically a problem, and we felt like bigtop
> would be a better home for it rather then perpetuate the bit-rot.
>
> Personally, I think we messed up by removing the code from our tree before
> bigtop has the necessary pieces in place. Since getting the code into
> bigtop is still progressing slowly[1], I've forked bigtop[2], added the
> patch and fixed some bugs that are currently in the patch.
>
> It's fairly easy to generate the packaging if you have some time to walk
> through it. I successfully installed and ran Accumulo 1.6.0 via Bigtop on
> Centos6.5 today.
>
> Rough outline:
>
> 1. Download the code from [2]
> 2. Look at the DEVNOTES [3] and install the prereqs (there may be more,
> but I tried to update the list)
> 3. Ensure Maven and Gradle are installed too
> 4. gradle hadoop-rpm
> 5. gradle zookeeper-rpm
> 6. gradle bigtop-jsvc
> 7. gradle bigtop-utils
> 8. gradle accumulo-rpm
>
> (check out `gradle tasks` for more info on steps 4-8).
>
> Then, install the packages which are placed in the build/ directory at the
> top level of the repo. `find` is your friend. the Accumulo RPMs and SRPMs
> both built successfully. I haven't tried building DEBs yet, so YMMV there.
> (hopefully will make some time later this week).
>
> - Josh
>
> [1] https://issues.apache.org/jira/browse/BIGTOP-1175
> [2] https://github.com/joshelser/bigtop/tree/accumulo-support
> [3] https://github.com/joshelser/bigtop/blob/accumulo-support/DEVNOTES
>

Re: RPMs/DEBs for Accumulo 1.6.0 via Bigtop

[Sean Busbey <bu...@cloudera.com>]

yes perfect!

Thanks Josh.


On Tue, Jun 24, 2014 at 11:45 AM, Josh Elser <jo...@gmail.com> wrote:

> I've updated the README in this fork to state explicitly that it is only a
> stopgap solution, that when the patch is applied to bigtop, this repo will
> be invalidated, and that it contains only material present in the current
> patch and recommended changes in the RB.
>
> Hope that addresses your underlying concerns.
>
>
> On 6/24/14, 12:50 AM, Josh Elser wrote:
>
>> Of course. I've been updating the review in the BIGTOP issue I linked in
>> my original email.
>>
>> Merely circumventing bigtop because they don't seem to be very active in
>> trying to get this merged (and not even putting it in master when the
>> time to merge does happen).
>>
>> On 6/24/14, 12:44 AM, Sean Busbey wrote:
>>
>>> Has the feedback for the bits you had to fix already been sent to bigtop?
>>>
>>>


-- 
Sean

Re: RPMs/DEBs for Accumulo 1.6.0 via Bigtop

[Josh Elser <jo...@gmail.com>]

I've updated the README in this fork to state explicitly that it is only 
a stopgap solution, that when the patch is applied to bigtop, this repo 
will be invalidated, and that it contains only material present in the 
current patch and recommended changes in the RB.

Hope that addresses your underlying concerns.

On 6/24/14, 12:50 AM, Josh Elser wrote:
> Of course. I've been updating the review in the BIGTOP issue I linked in
> my original email.
>
> Merely circumventing bigtop because they don't seem to be very active in
> trying to get this merged (and not even putting it in master when the
> time to merge does happen).
>
> On 6/24/14, 12:44 AM, Sean Busbey wrote:
>> Has the feedback for the bits you had to fix already been sent to bigtop?
>>

Re: RPMs/DEBs for Accumulo 1.6.0 via Bigtop

[Josh Elser <jo...@gmail.com>]

Of course. I've been updating the review in the BIGTOP issue I linked in 
my original email.

Merely circumventing bigtop because they don't seem to be very active in 
trying to get this merged (and not even putting it in master when the 
time to merge does happen).

On 6/24/14, 12:44 AM, Sean Busbey wrote:
> Has the feedback for the bits you had to fix already been sent to bigtop?
>

Re: RPMs/DEBs for Accumulo 1.6.0 via Bigtop

[Sean Busbey <bu...@cloudera.com>]

Has the feedback for the bits you had to fix already been sent to bigtop?

-- 
Sean
On Jun 23, 2014 11:32 PM, "Josh Elser" <jo...@gmail.com> wrote:

> One rather user-facing change that we made with 1.6.0 was the removal of
> the RPM and DEB packaging from our tree. We did this because maintenance of
> that packaging code was historically a problem, and we felt like bigtop
> would be a better home for it rather then perpetuate the bit-rot.
>
> Personally, I think we messed up by removing the code from our tree before
> bigtop has the necessary pieces in place. Since getting the code into
> bigtop is still progressing slowly[1], I've forked bigtop[2], added the
> patch and fixed some bugs that are currently in the patch.
>
> It's fairly easy to generate the packaging if you have some time to walk
> through it. I successfully installed and ran Accumulo 1.6.0 via Bigtop on
> Centos6.5 today.
>
> Rough outline:
>
> 1. Download the code from [2]
> 2. Look at the DEVNOTES [3] and install the prereqs (there may be more,
> but I tried to update the list)
> 3. Ensure Maven and Gradle are installed too
> 4. gradle hadoop-rpm
> 5. gradle zookeeper-rpm
> 6. gradle bigtop-jsvc
> 7. gradle bigtop-utils
> 8. gradle accumulo-rpm
>
> (check out `gradle tasks` for more info on steps 4-8).
>
> Then, install the packages which are placed in the build/ directory at the
> top level of the repo. `find` is your friend. the Accumulo RPMs and SRPMs
> both built successfully. I haven't tried building DEBs yet, so YMMV there.
> (hopefully will make some time later this week).
>
> - Josh
>
> [1] https://issues.apache.org/jira/browse/BIGTOP-1175
> [2] https://github.com/joshelser/bigtop/tree/accumulo-support
> [3] https://github.com/joshelser/bigtop/blob/accumulo-support/DEVNOTES
>