Whats a good venue to share bigtop examples ? blog? code? g+ hangout?

[Jay Vyas <ja...@gmail.com>]

Hi folks.  I was considering sharing some ideas / scripts about how we use
bigtop to test stuff either as code (i.e shell script which invokes various
maven tasks and copies files) to an examples/ folder in the main repo, or
else, write a bigtop blog post (i.e. that walks through the script and
explains the tasks).

Not sure if its worth an "official" code commit, or an official blog post
or wether I should just dump it on my personal blog and move on ...  But
let me know ! :)... Here is the basic contents of what the blog post or
example script will walk through.

0) Exports environmental variables

1) builds test-artifacts using standard maven online snapshot mode to get
dependencies and stuff.

2) then rebuilds the artifacts in offline mode to overlay our test-artifact
customizations on top of the work done in the previous step.

3) cp's some pom.xml files into the test-execution/test-artifacts folders,
including a custom iTest groovy snippet to run some pig shell tests.

4) starts flume

5) runs the test-execution task

Overall, it took a while for me to get it fine tuned to be a good cluster
test, but now I'm quite happy with it.

I think it could be a useful artifact to anyone that wants to run bigtop
smoke tests against a non standard hadoop deployment (i.e. anyone like me
... :) )

I'm leaning towards sharing this as a blog post since the script is built
to run against a heavily customized bigtop fork of mine (
github.com/jayunit100/bigtop) which is quite raw and needs some cleanup....
but if someone in the bigtop community wants to collaborate, maybe we can
streamline the example script so that people can use it as a template for
customizing bigtop smokes to their own needs.

Another option, possibly, is we could have a google hangout in the bigtop
community where we share the way we use bigtop to do different tasks and
provide each other with feedback.

-- 
Jay Vyas
http://jayunit100.blogspot.co <http://jayunit100.blogspot.com>

Re: Whats a good venue to share bigtop examples ? blog? code? g+ hangout?

[Konstantin Boudnik <co...@apache.org>]

Wiki sounds like a good idea for it. Why don't you post it there and we can
polish/improve it if needed?

On Wed, Sep 04, 2013 at 06:23PM, Jay Vyas wrote:
> Hi folks.  I was considering sharing some ideas / scripts about how we use
> bigtop to test stuff either as code (i.e shell script which invokes various
> maven tasks and copies files) to an examples/ folder in the main repo, or
> else, write a bigtop blog post (i.e. that walks through the script and
> explains the tasks).
> 
> Not sure if its worth an "official" code commit, or an official blog post
> or wether I should just dump it on my personal blog and move on ...  But
> let me know ! :)... Here is the basic contents of what the blog post or
> example script will walk through.
> 
> 0) Exports environmental variables
> 
> 1) builds test-artifacts using standard maven online snapshot mode to get
> dependencies and stuff.
> 
> 2) then rebuilds the artifacts in offline mode to overlay our test-artifact
> customizations on top of the work done in the previous step.
> 
> 3) cp's some pom.xml files into the test-execution/test-artifacts folders,
> including a custom iTest groovy snippet to run some pig shell tests.
> 
> 4) starts flume
> 
> 5) runs the test-execution task
> 
> Overall, it took a while for me to get it fine tuned to be a good cluster
> test, but now I'm quite happy with it.
> 
> I think it could be a useful artifact to anyone that wants to run bigtop
> smoke tests against a non standard hadoop deployment (i.e. anyone like me
> ... :) )
> 
> I'm leaning towards sharing this as a blog post since the script is built
> to run against a heavily customized bigtop fork of mine (
> github.com/jayunit100/bigtop) which is quite raw and needs some cleanup....
> but if someone in the bigtop community wants to collaborate, maybe we can
> streamline the example script so that people can use it as a template for
> customizing bigtop smokes to their own needs.
> 
> Another option, possibly, is we could have a google hangout in the bigtop
> community where we share the way we use bigtop to do different tasks and
> provide each other with feedback.
> 
> -- 
> Jay Vyas
> http://jayunit100.blogspot.co <http://jayunit100.blogspot.com>