You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bigtop.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/12/03 00:26:59 UTC
[jira] [Commented] (BIGTOP-2617) refresh juju spark-processing
bundle
[ https://issues.apache.org/jira/browse/BIGTOP-2617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15716960#comment-15716960 ]
ASF GitHub Bot commented on BIGTOP-2617:
----------------------------------------
GitHub user kwmonroe opened a pull request:
https://github.com/apache/bigtop/pull/168
BIGTOP-2617: refresh juju spark-processing bundle
Similar to BIGTOP-2616, we've made improvements to the spark-processing bundle that are ready for review and inclusion upstream:
- update readme for consistency with other bundles as well as calling out new sparkbench capabilities
- refresh bundled charms, including the use of bigtop-zookeeper
- reduce spark machine count (2 units instead of 3, which still allows HA without as much $$$ in cloud resources)
- ganglia/rsyslog have been updated from trusty to xenial
- add new machine constraints since aws now defaults to an 8gb root disk (we want 32g at a minimum)
- run the bigtop smoke tests for spark/zookeeper
- ensure the proper amulet test harness is included in the bundle tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/juju-solutions/bigtop bug/sp-cwr-fixes
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/bigtop/pull/168.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #168
----
commit 6c26d315e74ffddfa61dad7a8aa9bb0251706bd0
Author: Kevin W Monroe <ke...@canonical.com>
Date: 2016-10-28T15:14:48Z
reduce machine count (2 sparks instead of 3); refresh ganglia/rsyslog; use bigtop zk; bump amulet timeout to an hour
commit 56f03b60aa3f438aa848aa7a0087816f6cb274d5
Author: Kevin W Monroe <ke...@canonical.com>
Date: 2016-10-28T16:58:09Z
use charm smoke tests in our bundle test
commit c02b8f8286952ad21a4ea9c1df63e035afea6cff
Author: Kevin W Monroe <ke...@canonical.com>
Date: 2016-10-28T17:26:37Z
bump spark revno (longer test timeout)
commit 62e66da9f08e821c2a8931f1e9dda8361fe32b0d
Author: Kevin W Monroe <ke...@canonical.com>
Date: 2016-10-28T21:11:00Z
bump revno to pick up spark and zk test fixes
commit 78c6d20b6b00eeab5912ae3f7b6279e8303488dc
Author: Kevin W Monroe <ke...@canonical.com>
Date: 2016-10-28T21:12:17Z
fix pgrep
commit 1fd7e88230a4141d046db9713a71ec0bd89a96e6
Author: Kevin W Monroe <ke...@canonical.com>
Date: 2016-10-28T21:40:36Z
pickup zk-test fix from zk
commit 32ac3e230c01ba0bfba373b76d6bc143f381b3b6
Author: Kevin W Monroe <ke...@canonical.com>
Date: 2016-10-28T23:12:21Z
refresh readme to be consistent with other bundles; add zk<->ganglia monitoring; add a working ganglia unit
commit ffafa4f10cf0c15a1a910d8abcc7a6ef3f97e279
Author: Kevin W Monroe <ke...@canonical.com>
Date: 2016-10-31T16:41:15Z
update smoke failure message
commit 37b8387948834a7ecb14f5b1cddcd84e572cd300
Author: Kevin W Monroe <ke...@canonical.com>
Date: 2016-10-31T22:31:02Z
bump spark revno for spark-bench fixes
commit 412de035785fe0cc863714de7c985d23ef832ab1
Author: Kevin W Monroe <ke...@canonical.com>
Date: 2016-11-02T04:23:17Z
refresh spark charm and readme for new sparkbench suite
commit ee85b893f47448bfcbca787fe7d81f8812d0647b
Author: Kevin W Monroe <ke...@canonical.com>
Date: 2016-11-02T22:46:32Z
charm revision bump
- new ganglia charms report app version and fix '-broken' hook failures
- new spark charm correctly runs sparkbench regardless of spark exe mode
commit ac67e45672aa87b7db2d391cfc92201065067970
Author: Kevin W Monroe <ke...@canonical.com>
Date: 2016-11-04T22:50:41Z
bump spark rev to pick up testcase changes (be consistent with other bigtop charms) and app status reporting (no hard coded strings)
commit d6230776c24a564bdfdb9e471fee2b963edeb160
Author: Kevin W Monroe <ke...@canonical.com>
Date: 2016-11-05T22:37:57Z
bump spark rev for updated test case and sparkbench composite is now seconds
commit a72b3af219ecf21004c37d85f38c0872542ef242
Author: Kevin W Monroe <ke...@canonical.com>
Date: 2016-11-07T17:49:08Z
bump zk revno to pick up test rename and app version
commit c6a83e7596aec9b2316027eb19b693a684593f16
Author: Cory Johns <jo...@gmail.com>
Date: 2016-11-14T21:57:37Z
spark-processing tests.yaml should include PPA for Amulet
commit 73db79d1390b078ad1addd43f855adcb7aae5647
Author: Kevin W Monroe <ke...@canonical.com>
Date: 2016-11-15T18:48:43Z
bump rsyslog charm revs for lint fix
commit 1e053bb5cfe2041af6efbd63efb3e667df78a053
Author: Kevin W Monroe <ke...@canonical.com>
Date: 2016-11-23T01:36:35Z
charm rev bump, new constraints
- bump zk to latest (consistent bigtop_ prefix, readme tweaks)
- aws defaults to 8gb root filesystem; bump this to 20gb
commit 2afcc96b9eb3895af201f98f916a9bfa847949d3
Author: Kevin W Monroe <ke...@canonical.com>
Date: 2016-11-24T19:48:14Z
rev charms to pick up new swap handler
commit 7b63bdd05982c3e94bef30e6ae0442372584e4c6
Author: Kevin W Monroe <ke...@canonical.com>
Date: 2016-11-28T23:12:24Z
workaround azure root-disk: https://bugs.launchpad.net/juju/+bug/1645408
----
> refresh juju spark-processing bundle
> ------------------------------------
>
> Key: BIGTOP-2617
> URL: https://issues.apache.org/jira/browse/BIGTOP-2617
> Project: Bigtop
> Issue Type: Improvement
> Components: deployment
> Affects Versions: 1.1.0
> Reporter: Kevin W Monroe
> Assignee: Kevin W Monroe
> Priority: Minor
>
> Similar to #2616, we've made improvements to the spark-processing bundle that are ready for review and inclusion upstream:
> - refresh bundled charms, including the use of bigtop-zookeeper
> - reduce spark machine count (2 units instead of 3, which still allows HA without as much $$$ in cloud resources)
> - ganglia/rsyslog have been updated from trusty to xenial
> - add new machine constraints since aws now defaults to an 8gb root disk (we want 32g at a minimum)
> - run the bigtop smoke tests for spark/zookeeper
> - ensure the proper amulet test harness is included in the bundle tests
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)